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PREFACE* 



As a primary source of information to educators contemplating 
the use of programs of instruction, the U.S. Office of Education asked 
the Center for Programed Instruction, Inc. to conduct this Survey of 
the Use of Programed Instructional Materials in the Public Schools 



in the United States During the year 1961 -1962. A committee of edu- 
cators, whose schools have experimented with programed instruc- 
tion, assisted in determining the information which would be most 
useful to schools contemplating the use of programs at this early 
stage of the rapidly developing field. The committee consists of: 

J. Bernard Everett, Assistant Superintendent for Instruction, New- 
ton, Massachusetts*; Henry Brickell, Assistant Superintendent, Man- 
hasset Public Schools, Manhassiet, New York; George Fernandez, 
Board of Cooperative Educational Services of Ulster County, New 
Paltz, New York; Joseph O. Loretan, Associate Superintendent, 

Board of Education, New York City; Lloid Jones, Director of In- 
struction, Denver Public Schools, Denver, Colorado; Ernest Wein- 
rich, Board of Cooperative Educational Services, Huntington, New 
York; and Reginald Neuwien, Director, Administrative Research De- 
partment, Educational Research Council, Greater Cleveland, Ohio. 

Appreciation is herewith extended to these and the several 
thousand other busy educators whose data constitute the basis for 
this report. 



* Now on leave of absence in Afghanistan 
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This volume contains the results of the second half of the two 
part survey dealing with (1) available programed instructional ma- 
terials, and (2) the use of these materials in schools. As with Part i 

One this survey has jw?en carried out by the Center for Programed i 

Instruction, Inc., undei: contract from the U.S. Office of 
Education. 

Original publication plans called for a single volume with its 
two sections so arranged and cross-indexed that the reader might 
look up programs in the first section, and find publisher -supplied 
information (including preliminary testing results); in the second 
section the reader would be able to find information on how such 
programs are being used in schools, along with the reactions of 1 

teachers, administrators, students, etc., to these materials. j 

However, as a result of an increasing demand from educators j 

for a reliable list of programs, the decision was made to issue the 
information from publishers immediately so that it might serve as j 

a guide to schools purchasing materials for use during the 1962-1963 j 

school year. Therefore, Programs, *62 contains the information re- 
ceived from publishers in February on programs which these pub- 
lishers have acknowledged they will be able to supply to schools by 
September, 1962. Programs, *62 was published in the spring .of 1962 
by the U.S. Office of Education and is available through the Super- 
intendent of Documents as OE-34015 for $1.50. 

The second or “user” part of this guide makes up the present 
volume and will hopefully assist the potential user of programed 
materials in handling the parameters with which he is most apt 
to be concerned: cost, size of experimental population, most fre- 
quent methods of use, decisions involved, and above all the reac- 
tions of teachers, administrators, boards of education, parents, and 
even students to this early experimentation in the schools. This re- 
port is an attempt to assay the problems, advantages, and attitudes occur- 
ring in this very early period in the use of programed instruction. As such 
it is , of course, a survey of a very immature and experimental field; the 
first publishers into this market have borne a g . eat deal of the brunt 
of these early trials, as well as whatever financial advantages there 
may have been. It is safe to say that no one in the field of program- 
ing considers any present program “good* because most of us are 
beginning to be aware of the potential quality and improvements that 
will come. This volume may provide the potential user a picture 
which will encourage further experimentation, and may even pro- 
vide specific details around which or with which to frame his own 
experimentation. A major purpose of the survey and this report 
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Three questionnaires (Appendix A) were developed with the as- 
sistance of the advisory committee who had guided the selection of 
pertinent information to be presented in Programs ’62 and whose 
assistance is acknowledged in the Preface of the present volume. 
The individual forms were tried out on school administrators and 
program users who were not involved in the original information 
compilation, and were revised in accordance with their suggestions. 
The results still represent something of a compromise between the 
volume of information desired and the likelihood of obtaining any re- 
turn whatsoever from busy administrators and teachers. 

The almost 15,000 U.S. school superintendents listed by the Of- 
fice of Education received a covering letter and one copy each of 
the three questionnaires, “Non-user”, “User”, and “Individual 
Program Usage”. Superintendents with little or no experience with 
programed instruction were asked to complete and return at least 
parts of the Nonlaser form; recipients whose systems were using 
programs were asked to complete the User form and have someone 
directly involved return a completed Individual Program Usage 
sheet for each program. Returns were accepted, and follow-ups 
were attempted, through June of this year. 

Over two thousand school superintendents completed and re- 
turned one or more of the forms sent them. While it is probably 
not safe to make assumptions as to the completeness of the sample 
involved, or its representative nature (schools least involved in 
programed instruction would have a tendency not to respond), the 
present sampling is nevertheless presented to supply the sort of in- 
formation regularly requested by school systems, i.e., “What is 
being done in schools around the country?” This report is presented 
not as an accurate picture of all ways that programs have been 
used, nor as a complete sampling of subject matter or location of 
program use. The diversity of the histories sampled, however, 
should be sufficient to provide perspective and even encouragement for 
other school administrators interested in using programed materials. 

Geographically, the sample seems reasonably widespread; cer- 
tain concentrations of program availability (and nearby assistance 
in getting started) are apparent in the Northeast, the Midwest, and the 
Southwest. While a complete geographical breakdown was not per- 
mitted in the coding and processing of this first survey, the plan isto 
make some provision for this aspect of usage reporting in the future . 
Many other refinements will be suggested to readers of the present 
effort; please forward all such suggestions to the Center. 

The Center would like to go further and request that schools, 
particularly those involved in research with programs, send along any 
reports, even informally, that may be of assistance to others as 
program usage grows. 

Local appreciation is due to Carol Christman, Helen Geiger and 
Barbara Haas of the Center staff for their considerable assistance 
in assembling this manuscript. 

Lincoln F. Hanson 

The Center for Programed Instruction, Inc. 

365 West End Avenue 
New York 24, N.Y. 

July 1962 
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b. STEMS USING PROGRAMED INSTRUCTIONAL 
MATERIALS and 

SYSTEMS having LITTLE or NO EXPERIENCE 
with PROGRAMED INSTRUCTION 

GENERAL FAMILIARITY WITH THE FIELD 



As an introductory item in the survey, an attempt was made to 
assess just how much (or little) the non-users of programed instruc- 
tional materials might be acquainted with the field. To this end the 
initial question for this group asked three critical items designed to 
assist respondents in knowing how far to pursue in their responses, 
as well as to give us an indication of a kind of base level of sophis- 
tication of the respondents across the country. It must be supposed 
that an even larger proportion of the non-responding recipients are , 
included at or below this level of knowledge. 

Non-users were asked if they were familiar with terms such as 
Programed Instruction, Teaching Machines, Programed Learning, 
Automatic Teaching, etc. They were next asked whether they had 
ever read an article on programed instruction, and third, they were 
asked if they had ever seen a teaching machine. Respondents who 
answered “yes* to any of these three questions were asked to con- 
tinue to fill out the non-user form; those who responded “no* to 
all three questions were asked to complete only the final (identifica- 
tion) section of the sheet and return it directly. The results of this 
“screening* trio are presented in Figure 1. Responses to a subse- 
quent, but related, question, “Have you examined a program?” are 
included in this figure. 

Early in the examination of the returns of the “non-users* 
(superintendents whose systems had used little or no programed in- 
struction as of the date of the questionnaire early this spring), an in- 
teresting anomaly occured: 84% of the population indicated that they 
were familiar with the terms programed instruction, teaching ma- 
chines, learning machines, etc ., yet 90% had read an article about pro- 
gramed instruction. Whether this merely means that people took the 
term “familiar* in a more complete sense than intended, is hard to 
know; at any rate the degree of casual knowledge about programed 
instruction was surprisingly high in our sample. An obvious corol- 
lary, of course, is that those who did not return the questionnaires 
may very well constitute a significant section of the population to 



GENERAL FAMILIARITY WITH THE FIELD 
(Asked only of non-users) 



1. Non-Users 

Familiar with terms: Programed 
Instruction, Teaching Machine, etc. 

Read an article on Programed 
Instruction 

Seen a teaching machine 
Examined a Program 





whom these terms were not familiar and who have not read about 
programed instruction. The percentages represented in the fol- 
lowing pages should not be considered more than an approximate 
representation of the larger public school population. 

In this same general information category, 73% of the popula- 
tion have actually seen a teaching machine (whether this means “in 
person” or in a photograph is difficult to say) . A parenthetical 
point of interest to those in the field is that even though 73% of the 
population have seen a machine, only 44% have actually examined a 
program. 



* The absolute numbers of survey returns are not strict- 
ly comparable. For all intents and purposes, the propor- 
tions of each response are the comparisons which most 
educators will find of interest, and therefore in the discus- 
sion of the several figures that follow, “non-user” cmd 
“user” percentages will be compared without reference to 
the absolute “N’s* involved. Many items on the question- 
naire contained categories which were not mutually exclu- 
sive. Multiple categories checked obviously produce per- 
centage totals which are sometimes well over 100. 
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SOURCE and EXTENT of INFORMATION 



2a. Initial Source of Information 
Newspapers & magazines 



Professional publications 



Colleague 



Q 2.S 50 75 IQO% 



Lay members of community 



Salesmen 



Other 



Omit 




(users) 

( non-users ) 



On the matter of initial sources of information about programed 
instruction there are reasonable parallels between the users and the 
non-users. Educators seem to get their information primarily from 
professional publications; over 70% of both groups of respondents 
agreed in this regard. A larger proportion of the non-users 5 got + 
their initial contact from newspapers and magazines 32% vs * 17 
While the users seem to pass on information by word of mouth (al- 
most twice as many people got their first contact from a colleague 
among the users as among the non-users), very small portions 
of each group admit to receiving their first progra^ mformation 
from members of the lay community. Non-users got their initial in 
formation from salesmen about twice as often as the users did, 
miscellaneous sources were about the same in both groups. 

The information on periodicals read by both users and non 
users reflects an arbitrary selection of publications known to in- 
clude articles on or discussion of programed instruction, and there 
fore the miscellaneous category may in fact include journals and 
periodicals read to a greater degree than shown. 



* see Appendix C 
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SOURCE and EXTENT of INFORMATION (continued) 

2b. Periodicals Read 



AVI 



AID 



Automated Teaching Bulletin 
Programed Instruction 
J. of Educ, Psychology 



Others 



Omit 




_Z5 lflo% 



(users) 

(non-users) 



2c. Books Read 
Galanter 



Lumsdaine & Glaser 



Cram 



Stolurow 



Markle 



Other 



Omit 




1 
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SOURCE and EXTENT of INFORMATION (continued) 



Audio Visual Instruction is apparently read by 



over half of the users and at least a third of the non-users respond- 
ing. AID , a publication containing general information on Automated 
Instructional Devices produced by INRAD, Lubbock, Texas, was read 
by some 20% of the users and apparently has not gotten around near- 
ly so much to the non-users. The Rheem -Califone Automated Teach- 
ing Bulletin , now out of business but to be replaced by the Journal 



of Programed Instruction , was read by some 25% of the users and a 
small group of non-users. Programed Instruction , the bi-monthly 
bulletin put out by the Center for Programed Instruction under Car- 
negie grant, and originally sent free to all interested educators, was 
read by 55% of the users and only 24% of the non-users. The Journal 
of Educational Psychology is also read by about twice as many users 
(33%) as nonrusers (16%-17%). There is a small scattering of other 
periodicals containing information on programed instruction which 
is read by both categories. 

Books read are very much a function of the time at which the 
questionnaire arrived and the pattern would undoubtedly have chang- 
ed with the many new books which have come out since February, 
but as of early spring, 1962, Galanter’ s book on Automatic Teaching 1 
(reports of an early meeting on Automated Instruction held at the 
University of Pennsylvania in 1958) has been read by 20% of the 
users and a very small percentage of non-users. Lumsdaine 



and Glaser’s compendium 2 of early papers and reports 



(including some from the Galanter book) has been read by over 50% 
of the users and 18% of the non-users. Cram’s little paperback^ il- 
lustrating both intrinsic and linear programed instruction has been 
read by twice as many users as non- users; the proportions being 
20% and 10%, roughly. Stolurow’s Cooperative Research survey’ 
was quite fresh on the market and had been read only by small 
numbers of both groups. The Markle et al., primer on programed 
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Galanter, Eugene (Editor). Automatic Teaching, The State of 
the Art , John Wiley & Sons, New York, 1959. 

Lumsdaine, A. A. and Glaser, Robert (Editors). Teaching 
Machines and Programmed Learning: A Source Book, NEA 
Department of Audio-Visual Instruction, Washington, D.C., 1960. 
Cram, David. Explaining Teaching Machines and Programming, 
Fearon Publishers, San Francisco, 1961. 

Stolurow, Lawrence M. Teaching by Machine , OE -34010, Co- 
operative Research Monograph No. 6, U.S. Department of 
Health, Education, and Welfare, Washington 25, D.C., 1961. 
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SOURCE and EXTENT of INFORMATION (continued) j 

instruction^ had also been read by 20% of the users and 10% of the i, 

non-users. Once more there were a few miscellaneous books not j 

included on our list, such as Mager’s Preparing Objectives for Pro - ! 

grammed Instruction^ and Lysaught’s Programmed Learning .^ 



DOOR to DOOR SALESMEN 

A single question was included to sample user acquaintance with 
the phenomenon of machines being sold from door to door. Specifically, 
the question read, “How many parents do you know of who have bought 
such materials?” Of 209 respondents, eight said “none,” 152 said they 
knew of “a handful” that had been bought, and 49 said “a considerable 
number” had been bought. Parents, too, are an experimenting popula- 
tion and are obviously trying these devices on a scattered basis. These 
figures indicate that home utilization of programed instruction is not 
overwhelmingly large. The 400,000 programs which publishers report 
sold to homes, while impressive as an absolute number, mean that 
these material have touched perhaps l%of the nation’s homes. 



j 



5 Markle, Susan M., Eigen, Lewis D., and Komoski, P. Kenneth. 

A Programed Primer on Programing , Center for Programed j 

Instruction, Inc., New York, 1961. j 

6 Mager, Robert F. Preparing Objectives for Programmed In - ] 

struction , Fearon Publishers, San Francisco, 1961. f 

7 Lysaught, Jerome P. j Programmed Learning: Evolving Princi - 1 

pies and Industrial Applications , The Foundation for Research f 

on Human Behavior, Ann Arbor, Mich., 1961. j 
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STEPS TAKEN and PLANNED 



3a. Steps Taken 

System conference called 

School meeting called 

Planning group set up 

Appointed program director 

Staff sent to workshop 

Utilized consultants 

Obtained program samples 

Informed lay groups 1 

Other 
On^it. 



0 

& 




Of considerably greater interest to most readers in the educa- 
tional field will be the matter of the steps in the use of programed 
instruction actually taken or planned in both non-using and using 
school systems. Data from these two questions are contained m 
Figure 3. A number of schools have called a system wide school 
conference, with the users exceeding the non-users by a factor of 2. 
In similar fashion school meetings have been called more often by 
users than by non-users. The non-users drop far behind in the 
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STEPS TAKEN and PLANNED (continued) 



3b. Steps Planned 



zs so yy < 00% 



Call system conference 



Call school meeting 



Set up planning group 

. 

Appoint program dir 



Send staff to workshop 



ector 



Utilize consultants 



Obtain program samples 



Inform lay groups 



Other 



(users) 

( non-users) 



Omit 



matter of establishing a planning group, indicating perhaps that one 
of the ways in which new media are most successfully involved in 
the school system occurs when a group of interested teachers are 
given the guidance and responsibility to do their own investigating 
for their own particular situation. Planning groups are apparently 
a useful approach in this early stage of programing; the non-using 
group indicated plans to establish planning groups. 

A very small proportion of schools either use or plan to appoint 
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STEPS TAKEN and PLANNED (continued) 



a program director. Sending staff to programing workshops, on the 
other hand, ranks about the same as planning groups with the users 
and again is an activity planned to be continued in the future. Those 
who have not yet used much programed instruction, of course, lag 
in both categories hare. Thirty-four per cent of the group now using 
programed instruction have employed consultants and almost as 
many plan to continue the use of consultants in the future. Even of 
the non-users almost 12% have used consultants and 18% plan to use 
consultants in the future. The activity on which there appears to be 
the greatest agreement is obtaining samples of programs. Over 80% 
of the using group have obtained samples, and 43% are planning to 
obtain (more) samples; 35% of the group who are not using programs 
already have samples and plan to continue to obtain samples. The 
last step in program usage concerns the number of Schools who have 
taken steps to inform lay groups, parents, etc., about programed in- 
struction. It is reasonable to assume that this activity can be an im- 
portant step in preparing the community for educational change. 

The replies to this question contain one of the most significant dif- 
ferences between the users and non-users of programed instruction. 
Almost half of the group who are using programed instruction have 
taken such steps; whereas, only 8% of the group who are not using 
programed instruction have taken this step in good community re- 
lations. The users’ plans to inform lay groups (32%) also run well 
ahead of the non-using group (14%). 
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PATTERNS of USE 



4a. Usage Made of Programs 

Tried on -individual pupils 



Tried on groups of pupils 
Developed own programs 



Tested experimental hypotheses 



Other 



Omit 




4b. Planned Usage of Programs 
Try on individual pupils 



O B so 25 jO fl% 



Try on groups of pupils 
Develop own programs 



Test experimental hypotheses 



Other 



Omit 




Forty-six per cent of the respondents have tried programs on 
individual students, 80% on groups of students; 31% plan to try pro- 
grams on individual students, and 52% plan to work with groups. A 
surprisingly large number of schools (17%) are developing their own 
programs. The same proportion of respondents, although not in all 
cases the same individuals, is planning to develop local programs. 

A slightly smaller group (14%) is testing experimental hypotheses, 
a phrase which may mean a variety of things, but a reasonable num- 
ber do indicate an attempt at, or at least an interest in, a rigorous 
approach to research. Almost 20% plan to test experimental hy- 
potheses in future program usage. 
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PATTERNS of USE 



4c. Programs Used 

On limited trial basis 

For regular instruction 

With controls and research design 

Omit 



O Z5 SO 75 tOO% 




t 



Very few (3%) of the reporting schools omitted this question on 
present usage; looking ahead, however, 30% of the same schools 
omitted any “planned usage.” The distribution of the percentages in 
“planned use” is obviously not the same. Of those school systems 
now using programs either with individuals or groups 57% have said 
they are using programed instructional materials “on a limited trial 
basis.” Forty -three per cent of these schools are using programed 
materials for regular instruction. Almost 25% indicate that they are 
using programs with controls and with an experimental design.^ 



8 



Programed instruction provides an unusual opportunity for 
learner feedback in its direct (and recordable) contact with 
the student. An almost unique and quite new means of mani- 
pulating variables in the instructional process is now avail- 
able to assist experimentally minded educators. 
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INTENTS and PURPOSES 



5a. 



Intended vs. Actual Use of Programs 
Without machines 

With machines 

Undecided 




5b. Purpose 



Remedial Work 



Regular instruction 



Enrichment 



Not at all 
Omit 



While 7% plan to use programs alone, and 5% contemplate using 
programs in machines, 70% are presently undecided as to how they 
will use programs in the future. In the purposes for which they con- 
sider using programs, the non-users indicate that about 30% of them 
plan to use programs for remedial work; a mere 20% plan to use 
programs for regular instruction; and almost 35% plan to use pro- 
grams for enrichment. In other words, the peripheral combination 
of remedial work and enrichment together consitutes the principal 
use that the present non- users have in mind for programed instruc- 
tion. 

The user distribution is slightly different. While a little over 
half of the users have been using programs for remedial work, over 





INTENTS and PURPOSES (continued) 

two-thirds have been using programed instruction for regular in- 
struction and some 60 % have been using programed instruction for 
enrichment. While the appreciation of the possibilities of programs 
for regular instruction may grow with use, the total pattern at pres- 
ent still heavily favors the peripheral uses of programed instruction. 
These response categories are not mutually exclusive; many sys- 
tems are using programs for remedial, regular, and enrichment in- 
struction. 
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FINANCING PROGRAM USE; PRESENT and PLANNED 



6a. Present Funding 

Regular school budget 
Extra local funds 
State funds 
Federal funds 
Publisher 
Other 

6b. Future Funding 

Regular school budget 

Extra local funds 

State funds 

Federal funds 

Publisher 

Other 



© ts so rs ioo% 






MMMiMiMllfliflii^^ 



FINANCING PROGRAM USE; PRESENT and PLANNED (continued) 



Among the administrative details queried were methods of fi- 
nancing the use of programed instruction. Almost 90% of the users 
of programed instruction have operated under their regular budget 
and furthermore, they plan to continue using these materials within 
their regular budget! The total number of school systems whose use 
of programs has been financed with extra local, state, or federal 
funds, publisher support, or by any other means is still less than 
25% of all of the respondents. Interestingly enough this pattern also 
shows in the plans of the group that has not yet used programed in- 
struction —70% of them plan to use their regular budget to finance 
any future use of programed instruction, and only 12% have any ex- 
pectation of extra funding from [local, state, federal, publisher or] 
any other source. 9 

In predicting financial problems in the future use of programs, 
while 21% of the using school systems reported that there would be 
more problems than usual, the considerable majority (70%) indicate 
that they do not expect more than the usual financial problems. A 
very small group (9%) omits this prediction, and an interestingly 
large proportion (16%) commented on financial problems in the use 
of programed instruction. 



On this matter of other sources of income to support the use of 
programs, several schools wrote that they charge the student 
for the program and thus keep program purchases independent 
of their budget. While this practice enables these schools to use 
programs slightly more freely, it raises the question of a public 
school not supplying these materials without charge to all stud- 
dents. 
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7. First Use Traceable To: 
Teacher 

Curriculum coordinator 
A-V director 
Principal 

Other administrator 
Interested layman 
Other 







In the schools that are using programed instruction 22% ac- 
knowledge that a teacher was responsible for the first program use; 
the curriculum coordinator was the initiator 21% of the time; audio- 
visual directors, 5%; school principals, 20%; and other administra- 
tors (this usually means the superintendent or his associates) ac- 
count for 32% of first use. A few systems indicate a debt to inter- 
ested laymen for their initial activating force, and there is included 
a sprinkling of other non -administrative categories, consultants, etc. 
TIME 

Teachers Using or Developing Programs 
On own time 
On released time 
In paid overtime 
In paid vacations 
School or regular time 
Omit 
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TIME (continued) 



On the matter of providing time for teachers who are using or 
developing programed materials, 50% of the using schools did not 
answer this question at all. Of those who did respond, 44% indicated 
that teachers were using or developing materials on their own time. 
This response item “on own time* in the minds of those who as- 
sembled the original questionnaire, was intended to include any 
time squeezed out of the school schedule; nevertheless, some 4% 
of the group made a specific comment to the effect that “own time* 
was not a proper phrase and that “ school time* was an additional 
category (reported above!). The second response provided was “re- 
leased time* which again might have conceivably taken care of the 
“school time.* At any rate 15% of the responding users indicated 
that they did release time. A very, very small proportion provided 
paid overtime (2.4%) or paid vacation time (4.3%) for developing 
programs. 
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TIME LAGS 



i 



Immediately 
1-2 months 
3-6 months 
1 year 
Over 1 year 
Omit 



Time: Contact 
to First Trials 




Time: Experimental 
to Classroom Use 

O 25 50% 



t 
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How long does it take to get programed instruction into opera- 
tion? Two categories were listed for the respondent schools: 

(1) Time from initial contact to first trials, (2) Time from experi- 
mental or initial trials to classroom use. 

One per cent put programed instruction into trial use imme- 
diately; 10% took up to two months; 34% took three to six months; 

35% took closer to a year. Those who took over a year were about 

18 % 

°* Figures for the second unit of time, from experimental to class - 
room use, show that a reasonable number (20%) of the respondents 
apparently felt free to put programed materials to use in the class- 
room immediately, once they had tried them out. Six per cent took 
less than three months in addition to the experimental use, while 
an additional three to six months were taken by almost 25% of the 
group; almost a year from the experimental use to the classroom 
was taken by 19%; and over a year was taken by 7%. For the con- 
siderable 25% that did not check any experimental -to -classroom- -use 
time interval, the assumption is that this omission represents the 
schools who are not yet using the programs on other than a trial 
basis, that is, the group who have already taken their three months 
or a year to get into the experimental stage and who have not 
adopted programs for the classroom at this date. 

Summing up these two time factors, it would look as though 
some interval between three months and a year was the modal 
time from initial contact to at least some classroom use. 
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EVALUATION 



10. System Ability to Evaluate Material 

Non-programed materials 
Yes 

No 

Omit 

Programed materials 
Yes 

No 

Omit 



Seventy -nine per cent of the system administrators expressed 
confidence in their own staff with the evaluation of conventional 
materials (14% did not and 8% omitted). For programed materials, 
a slightly lower confidence level was evidenced with 54% feeling 
that their own systems could adequately evaluate programed in- 
structional materials. Thirty -five per cent of the school adminis- 
trators indicated that they did not think their staff was yet ade- 
quately prepared to evaluate programed materials, and 12% omitted 
this half of the question. 
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11. System Reactions 



Teachers 

Enthusiastic 



Favorable 



Neutral 



Opposed 



Strongly opp. 



to 

o 



Omit 



Students 

Enthusiastic 



Favorable 



Neutral 



Opposed 
Strongly opp. 



Omit 




£ 0 /J Administration 
Enthusiastic 



Favorable 



Neutral 



Opposed 



Omit 




50 % Parents 

Enthusiastic 



Favorable 



Neutral 



Opposed 
Strongly opp. 



Omit 




s_o7, Bd. of Education q 
Enthusiastic 



Favorable 



Neutral 




Strongly opp. 



Opposed 



Strongly opp. 



Omit 
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SYSTEM REACTIONS 



SYSTEM REACTIONS (continued) 



Preliminary and general evaluation by school systems using 
programed materials is shown in Fig. 11. Individual program usage 
evaluations by teachers (or more local administration) are pre- 
sented later (Fig. 20). In almost all cases the present information 
represents the administrative evaluation of local reaction on a five- 
point scale ranging from * enthusiastic* through “favorable,” “neu- 
tral,” and “opposed,” to “strongly opposed.” 10 The respondents in- 
dicate a reaction from the teaching staff as “ enthusiastic” in 22% 
of the using systems; 55% report a “favorable” reaction; 15% re- 
port “neutral”; 5% are “opposed” and one-half of 1% re- 
port a “strongly opposed” teacher reaction. (This last figure indi- 
cates that in only one school system the predominant teacher reac - 
tion was felt to be “strongly opposed”). 

The administration rates its own reaction as slightly more 
enthusiastic; almost 30% of the school systems using programed 
instruction report a reaction of “enthusiastic.” As with the teach- 
ers, 53% report a “favorable” reaction; a small group reports 
“neutral,” and only 1% of administrators are “opposed.” For par- 
ents, a smaller population is represented; apparently fewer school 
systems felt that they could speak for the parents. Twelve per cent 
of the parent reactions are reported as “enthusiastic”; 37% are 
“favorable”; 23% are “neutral”; and less than 3% are “opposed.” 

Boards of education, also somewhat less directly involved in 
the problems of program usage, are reported as showing the same 
general pattern: 12% “enthusiastic”; 45% “favorable”; 19% “neu- 
tral”; and only one board of education reported as being “opposed.” 
Student reactions resembled the pattern of their teachers more 
closely than any of the other groups: 24% “enthusiastic,” 48% “fav- 
orable,” 11% “neutral,” 5% “opposed,” and one group of students is 
reported as “ strongly opposed,” too. (This school system is not the 
same one in which the “strongly opposed” teacher reaction was re- 
ported). The only miscellaneous reaction reported as “strongly op- 
posed” was a group of “college math teachers in the area.” 



10 Where a range of reactions was checked only the average of 
the checked reactions was tabulated. This arbitrary technique 
may mean that the reader will wish to weigh the extremes 
more heavily than the data quoted. Specifically, for example, 
three systems indicated some teacher reaction as “strongly 
opposed.” Two of these also indicated a full range of all other 
attitudes up through “enthusiastic.” Only the one school sys- 
tem in which “ strongly opposed” was the predominant reaction 
shows up in the tabulation. Only one student reaction was 
checked as “strongly opposed.” 






INFORMATION WANTED 



■ ■} 

12 . Using Systems Getting Insufficient 
Information On: 



Other school usage 
Ways of utilizing 
Programs available 
Program constructing 
Research being done 
Omit 



One question was asked as a guide for future publications: 

“Which of the following are you not getting enough information 
about?*. Forty per cent of the respondents want more information 
on other school usage; 44% want information on ways of utilizing 1 

programs; over 50% of the group want information on available I 

programs; 32% are interested in more information on the construe- j 

tion of programs, while 55% of the respondents indicate a lack of s 

sufficient information about research being done in the field. 
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DISTRIBUTION by ENROLLMENT 






13. Enrollment of Respondent School Systems 



0-499 



500-999 



1000-4999 



25 SO 75 10 0 % 



5000 -J- 



(users ) 
(norr users) 



A slight differentiation in schools using and not using pro- 
gramed instruction turned up in the enrollment distributions of re- 
spondent school systems. While system enrollment of around 1,000- 
5,000 constitutes the same proportion of both non-using and using 
populations (37% and 38% respectively), the proportion of non using 
schools with less than 1,000 enrollment totals 42% of the responding 
group, whereas in the using population 32% of the responding schools 
are over 5,000 in size. 

For every responding school or system with an enrollment of 
500-1,000 students there were two schools under 500, there were 
four schools or systems with 1,000-5,000 students, and there were 
three schools or systems with more than 5,000 enrolled. In other 
words, in the ratio of 2: 1:4:3, the increasing size school systems 
may be evaluated against one another in terms of proportionate re- 
sponse. For example, on the matter of having a system-wide con- 
ference, it is quite clear that the proportion of schools in the 1,000- 
5,000 Class who have called a system -wide conference on program- 
ed instruction is greater than in any other single enrollment cate- 
gory of schools. A brief discussion of the data on size of school 
(or system enrollment) may assist the reader in selecting appro- 
priate action for his own school system. For a second example, in 
meetings within schools this same category of 1,000-5,000 size 
schools has the largest absolute number of responses checked, but 
the proportion by enrollment ratio is not as great as in the schools 
of over 5,000. Perhaps all three of these figures indicate that when 
school systems grow large enough, system conferences and school 
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DISTRIBUTION by ENROLLMENT (continued) 



meetings are not only more necessary, they are called! On the 
other hand, the smaller enrollment categories indicate that they 
plan to have school meetings in greater proportion than do the 
larger school systems. 

Planning groups, in the same vein, are a function of the larger 
school systems, in both present activity and intent ; the very largest 
school systems are clearly the strongest believers in planning 
groups. When the system is large enough, not only does it use 
school meetings and system -wide conferences, but apparently there 
is an operational need for a specific group (as the City of Phila- 
delphia has, for example) whose function is to pull together and 
disseminate information on programed instruction and to obtain 
and provide guidance in the selection and introduction of pro- 
gramed material into the schools. 

The school systems which have program directors are also, 
logically enough, those with larger enrollments. The proportion of 
systems who sent staff, or who plan to send staff, to a workshop 
follows the 2:1:4:3 ratio almost perfectly. In other words, the 
value of sending staff to workshops is appreciated and planned 
in direct proportion to the number and size of schools represented 
in the survey. In similar fashion samples have been and will be ob- 
tained across the board. Consultants are used and planned much 
more at the 5,000+ enrollment level, which makes sense financially, 
if no other way. The larger systems feel they can afford (or feel 
that they cannot afford not) to have a consultant to assist with their 
selection and early use of programed instructional materials. 

One of the more interesting breakdowns turns up in the matter 
of informing lay groups (parents, etc.) about this new activity and 
taking steps to keep them up to date. This activity exists and is 
planned in almost exact proportion to the size of the enrollment of 
the responding schools, as represented in the survey and while 
these figures are not large, their distribution suggests a wide- 
spread appreciation of the importance of keeping in touch with lay 
groups, regardless of school or system size. 

The largest school systems seem to put teachers on theif “own 
time" considerably more than the smaller ones. Whether teachers’ 
“own time" means fewer contact hours but more administrative 
duties or whether there is more true free time provided by the 
larger schools for program implementation and development is 
hard to determine. Certainly this is an optimistic explanation for 
the dominance of the large school systems in this category. 

In released time the schools follow the enrollment proportions 
represented in our sample very closely. Paid overtime and paid 



DISTRIBUTION by ENROLLMENT (continued) 



vacations seem to be a luxury only the larger school systems feel 
they can afford. H 

The first use of programed materials in the very small school 
systems is traceable largely to a teacher. In the 1,000-5,000 pupil 
systems also, teachers seem to have an influence in introducing 
materials. In relatively few of the larger schools does this influ- 
ence hold. The balance comes in two alternative categories; the 
curriculum coordinator and the audio-visual director now more than 
carry their weight in introducing programed instruction. In larger 
schools these two roles dominate in introducing programed material, 
but one must remember that schools under 500, for example, prob- 
ably have no separately designated curriculum coordinator or audio- 
visual director. 

Perhaps again reflecting the changing influence of the individual, 
the principal himself in the smaller school systems is apt to be the 
one to introduce programed instruction, whereas the proportion of 
principals who have introduced programed materials into larger 
school systems, and particularly in the largest category, occurs 
less frequently. There is consistency in this change of the princi- 
pal’s role in more formalized administrations; some “other ad- 
ministrator” (often the superintendent) was also responsible for the 
introduction of programed instruction in a greater proportion in the 
smaller school systems than in the larger ones. Laymen were in- 
strumental in introducing programed instruction only in school sys- 
tems of 1.000 or more pupils. This figure is consistent with Brickell’s 
report. ^ Specific instructional innovations are rarely demanded or ef- 
fected by the public or the board of education, but the demand 
for an improved outcome is an important concern of the board, and 
ultimately the lay public. Increasing public awareness of program- 
ing potential is apparently a worthwhile goal. Certainly, the public 
is concerned with the end result of introducing programed materials 
while the teacher and administrator are more immediately concern- 
ed with the details of the innovation process . 

The reactions of using systems, by enrollment, are given in 
Appendix B, Table II. In the smaller schools, the enthusiasm runs 
higher than the enrollment ratio (2: 1:4:3), while the opposition quite 
clearly shows up only in the larger school systems— no matter 
which the group reacting. 

11 It is interesting that the category not provided for, “school”or “reg- 
ular” time, assumed to have been covered under “own time” or “re- 
leased time” was frequently written in as an extra response category. 
The validity of this response as a real omission is supported 

by a proportionate representation in each category of school size. 

12 Brickell, Henry M. Organizing New York State for Educational 
Change, State Education Department, Albany, New York, 1961. 
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NUMBER and COST of PROGRAMS USED 
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15. Number of Copies 
Used 



Copy Cost 



zs 



1-5 



6-23 



24-49 
50-99 
100-299 
300 + 



_5P% 



Free 



J 



3.5 



507. 



Less than $1. 00 
$1. -2.99 



] 



$3. -4.99 
$5. -9.99 






$10. -15.00 
$15.00 -f 



The modal category of number of programs used is between 
24 and 29 copies (the category checked by 27% of the users). Eigh- 
teen per cent of the schools used less than five copies; 18% used 
between six and 23; above the mode, 16% used between 50 and 100 
copies; 15% used between 100 and 300; and 5% used 300 or more 
copies. 

Seven per cent of the program users obtained copies free. This 
figure includes early materials sent out by publishers for experi- 
mental purposes. Two per cent used programs which cost less than 
a dollar; 17% used programs that cost between three and five dollars; 
another 5% between five and ten dollars. The modal 10 to 15 dollars 
was spent for 37% of the programs and 9% of the respondents paid 
over 15 dollars per copy. Some of these “over 15 dollars” pro- 
grams involved hardware, although this is hard to detect from the 
present survey. Four per cent of the group responding indicated 
that they paid more than one price per program copy, in other 
words, they purchased some materials at one price and other ma- 
terials at a different price. 



PROGRAMERS and PROJECT DIRECTORS 



Local programing was usually done by a subject matter teacher. 
A few programs were written by a curriculum coordinator, 
and audio-visual specialist, and other miscellaneous categories. 
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PROGRAMERS and PROJECT DIRECTORS (continued) 

In a few cases a multiple effort was involved in the programing. 
Project direction shared a somewhat similar distribution: subject 
matter teachers directed most projects (23%), curriculum coordi- 
nators directed 9%, 1% were under audio-visual departments, and 
16% of the project directors came from diverse sources. Included 
in this group were principals, other administrators, school psychol- 
ogists, subject matter consultants, and faculty from nearby col- 
leges. Three per cent of these projects were under multiple direc- 
tion. 

Of the responding program users, 54% indicated that they were 
trying programs on a limited basis only, while 27% indicated that 
they were using programs as an integral part of the curriculum 
(an interesting contrast with the 43% figure reported on the returns 
from the system level). Twenty -one per cent are using programs 
“with controls and research design*; 5% checked “other* and 4% 
omitted the category completely. 
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Form of Presentation 
Horizontal text 

Vertical text 

In teaching machine 

Machine program w/o machine 

Auditory 

Multiple 

Other 



Supplementary Materials 




Regular text 
Moving picture 
Film strip 
Slides 

Overhead projector 

Tape recorder 

Phonograph 

Other 

Omit 











FORM of PRESENTATION: SUPPLEMENTARY MATERIALS 



\ 



FORM of PRESENTATION; 

SUPPLEMENTARY MATERIALS 

Only one school reports using programs on cards, 28% are 
using horizontal texts, 38% are using vertical texts, 12% use pro- 
grams in a teaching machine, while 7% used a machine program 
without the machine (easily possible with 8 1/2 by 11 sh ® et ®’ 
which can be used in several machines, with a simple mask or 
without any accessories); 2% of the programs use some auditory 
presentation; 7% are using multiple forms including audio and 
visual presentation; and 4% omitted the question. . . j n 

Most of the programs checked as being used by respondent 
this survey can be presented without machines. At least one school 
system, however, is using a reasonably elaborate (and ^pensive) 
projection device for individual program presentation. The 7% who 
indicate a multiple form of presentation may also account for som 
of the machines, which are needed to obtain combmations of pic- 

The next question inquired about supplementary materials used 
simultaneously with programs. Forty-five per cent no supple- 
mentary materials; 43% used a regular text along with the pro- 
gramed material. Motion pictures, film strips slides overhead 
projector, tape recorders, and phonographs total to less than 40% 
of the responses on supplementary material used, and many of 
those were used by the same person. 
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17. Program Use 



At Each Grade Level 
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In Each Mental Ability LeveP 



In Each Location 



With This Intent 



r Above average 
Average 
„ Below average 
Class 
Study hall 
Home 

I Remedial instruction 
Regular instruction 
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"As active supplement 
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GRADE LEVEL 



Forty-two per cent of the programs were used at or below the 
sixth grade; 31% of the programs were used by students in eighth 
grade and 42% in ninth grade (making this both the median and the 
modal grade level); 32% of the usage was in 10th grade; 25% in 11th 
grade; and 24% in 12th grade. The concentration of program usage 
is clearly at the high school level. The fact that percentages sur- 
pass 100 indicates that many programs were used at more than 
one grade level. 

Most of the schools involved used programs with only a few 
students. Categories checked run over 300, but the largest per- 
centage in any single category is the 11% who used between one 
and five students in the 10th grade. Very few schools used 300 or 
more students in their trials. Modal usage is five or six students. 
(Appendix A, “Individual Program Usage”) 



MENTAL ABILITY 

The mental ability levels of the students actually used by the 
reporting schools (58% above average; 54% average; and 35% below 
average) hardly constitute a normal distribution curve, although the 
intent expressed by individual program users is 23% remedial, 74% 
regular instruction and 24% enrichment, which is considerably 
closer to the “normal.” 



LOCATION 

Eighty -four per cent of the programs covered in the survey 
were used in class, 20% were used in study hall, and 22% were used 
at home. Close surveillance is the rule in program usage to date. 



ROLE of the TEACHER 



Sixty per cent of the program users saw the role of the teacher 
as an active supplement to the program. This “active supplement” 
presumably includes lectures, tests, talking about, and adding to the 
program in other ways. Forty-one per cent of the programs were 
used with the teacher “available for questions,” and only in 11% of 
the cases were teachers used as “proctors only.” Presumably most 
of these latter schools were involved in experimentation rather than 
regular instruction and were attempting to evaluate the program as 
an independent instructional agency (at least for that particular sec - 
tion of the subject matter). 
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18. Amount of School Schedule Devoted to Programed Instruction 

H our s / Day o zs s o% Days /Week o zs g o?. Scheduled Weeks o zs 
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SCHEDULE (continued) 



In the amount of school schedule devoted to programed instruc - j 

tion, the modal value of hours -per -day is between one half and one 
hour. Modal number of days -per -week is five, and scheduled weeks 
show a modal value of “over 32* (checked by 35% of the group). j 

An approximated median number of weeks amounts to between one 
and two semesters. j 



EVALUATION-PROGRAM SELECTION ! 

j 

Responses to the question on program selection indicate that ; 

11% of the users selected programs as the only ones available, 24% ■< 

of these respondents indicated that their programs were selected on 
the basis of publisher statements, programs were picked with the j 

aid and assistance of the curriculum coordinator bv 23% of the i 

group, 16% used the guidance of a subject matter consultant, and j 

42% relied upon the subject matter teacher in choosing the program. 

Twelve per cent used some other source, but unfortunately 90% of i 

the people answering this questionnaire did not indicate any basis 1 

for program choice; in other words the percentages involved in the 1 

distribution above reflect a number of cooperative choices, but may j 

not indicate the basis upon which the majority of decisions were 
made. j 
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EVALUATION-PROGRAM CONTENT & TESTING 



19. Evaluation of Content 0 
Dark ages 



as- 



Somewhat obsolete 



Reasonably up-to-date 
Modern 



Too advanced 



g 0 1 Tests Used 

Standardized 



Publishers 



Teacher constructed 

Other 

Omit 



A? 



50 % 



No one thought the content of any program deserved the cate- 
gory of “dark ages”; 6% allowed that the program they had used 
was “somewhat obsolete”; 37% thought their program “reasonably 
up-do-date”; 48% considered their program “modern”; and no one 
thought the program material “too advanced.” 

For their criterion testing, or the measure of what the students 
learned from the program, 37% of the respondents used a standard- 
ized test; 33% relied upon a publisher supplied test; and 40% used a 
teacher constructed test, 7% used another test and 11% omitted any 
response. 
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EVALUATION -OVERALL EFFICIENCY 



'20. Overall Efficiency 
Useless 



Adequate 



0 2* §PZ 



1 



Extremely worthwhile 



Omit 



No one considered the experience with programing “useless*; 
for 6% the experience was something between “useless* and “ade- 
quate*; 30% considered the programing experience “adequate*; and 
29% considered it something better than “adequate.* Eighteen per 
cent considered their program use as “extremely worthwhile*; 
while 2% checked more than one category. Fifteen per cent of the 
respondents did not check any evaluation of overall efficiency; many 
of these wrote comments such as “too early to judge,* and “one test 

is not enough.* 13 



13 Different school systems of similar size and type using the 
same program under similar circumstances come up with com- 
pletely different “evaluations.* This obviously reflects different 
standards, expectations, methods of judgment, tastes and the 
like. It also points up one of the major difficulties in designing 
“evaluative criteria* for programed instruction. It would fur- 
ther imply that caution should be used in allowing the “experi- 
ences of others* and “testimonials* to enter into the decision 
making process concerning programs. 
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PROGRAM INFORMATION WANTED 



21 . Information About a New 

Statement of objectives 
Content classification 
Sample frames 
Criterion frames 
Other information 



Program Would Include 




The last information question on this form elicited the fact 
that 34% of the respondents would like information on any new pro- 
gram to include a statement of the program objectives; 53% want 
some kind of content classification; 39% would like to see sample 
frames; 21% would like to have author -indicated criterion frames; 
and 10% would like some other form of discriminative guidance. 

Subject matter teachers filled out 23% of the individual program 
usage forms, curriculum coordinators 16%, audio-visual represen- 
tatives 3%, and 29% of the respondents fell into a wide group of 
miscellaneous categories. 
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PROGRAM USERS -by SIZE and TYPE of SCHOOL 
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Distribution of Program Usage 
by Enrollment 



0-499 
500-999 
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Omit 

by Type of System 
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PROGRAM USERS -by SIZE and TYPE of SCHOOL (continued) 



The distribution of actual program usage by enrollment and by 
type of school system is given in Figure 22. A more complete 
breakdown of usage responses, also by enrollment, is given in Ap- 
pendix B, Table HI. A few unusual proportions are worthy of men- 
tion. 

Smaller school systems are using programs in more than their 
enrollment proportion of “Trial Basis” and “Integral Part of the 
Curriculum,” while the larger schools are using a greater percentage 
of research design. The larger schools do not keep up their share 
of enrichment usage, have more active teacher supplement to the 
programs, but used a smaller proportion of subject matter teachers 
as their basis for selecting the program. Smaller schools, in pro- 
portion, more often considered programs “Modern,” while larger 
schools leaned more conservatively to “Up-to-Date.” 

In the choice of tests used to evaluate student progress with the 
programed material, large schools used standard and teacher -con- 
structed tests very heavily, but for some reason used less than 
their share of publisher -supplied tests. 

The smallest systems “went overboard” in judging the overall 
efficiency of program usage, their larger colleagues being some- 
what more conservative. In the final section of Table III, it is the 
larger schools who are most specific in their requests for new pro- 
gram information. A statement of the program objective, sample 
frames, and author-designated criterion frames are all high on 
their lists. 

A further breakdown of individual .program usage data by sub- 
ject matter is presented in Appendix B, Table IV. The proportion- 
ate number of copies in each subject matter is presented in the first 
section of Table IV. Local programed s title, project director and 
purpose and form of presentation are given for each subject mat- 
ter, as well as the number of programs used in each grade, and 
the student ability levels involved. 

Amount of school schedule is similarly presented for each sub- 
ject, as are the bases for initial selection and all the evaluations 
mentioned in the preceding section. A final breakdown shows the 
number of schools in each enrollment category using each subject. 
Small numbers in each cell and the relatively few programs avail- 
able this year make comparisons difficult if not meaningless. The 
breakdowns in Tables n, III, and IV are included principally so that 
cooperating schools will be able to place their usage in a larger 
perspective. 
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THE USAGE SURVEY COMPARED WITH PROGRAMS *62 
PROGRAMS USED and PROGRAMS AVAILABLE 



23. Subject Area 

l 1 Programs used (1961-62) 
HHI Programs available (in 1962) 



Grammar, usage, spelling 
Foreign languages 



O V 

LZ3 



y so ? c% 



i 



Mathematics 



Sciences 



Social sciences 



Other 



Li 

i 

. 



An interesting comparison is afforded by data on program sub- 
ject areas available in Programs, *62 and the present data on the 
subject matter of programs actually used. Programs in grammar, 
usage, and spelling constitute 17% of the programs now available 
and constituted 25% of the programs used. Programs in foreign lan- 
guage constitute 8% of the programs available, 4% of the programs 
used. Mathematics programs, which represent the majority of 34% 
of programs available, constituted an even larger proportion (61%) 
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PROGRAMS USED and PROGRAMS AVAILABLE (continued) 



of programs used as reported in the survey. Nineteen per cent of the 
programs now available are in science, but only 3% of those being 
used are science programs. Six per cent of the programs available 
and 3% of the programs used are in social science. All other cate- 
gories make up 15% of the programs available; only 3% of these other 
categories of programs were used by the schools reporting. In the 
case of mathematics and grammar, where the percentages used are 
actually in larger proportion than the percentages available, the few 
programs that were available a year or more ago dominate present 
usage. 



PROGRAM HOURS 



24. Hours of Programed Subject Matter 
I I Hours used 

Hours of programed material available 



Grammar, usage, spelling 



So % 



Modern language 



Mathematics 



Sciences 




Social sciences 



1 



Other 






In a comparison of available program hours with the number of 
program hours actually used this year, some approximations are 
represented in Figure 24. Grammar, usage, and spelling constitute 
8% of programed hours (to be) available and constituted 16% of the 
hours actually used. Modern language shows a very nice consistency 
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PROGRAM HOURS (continued) 



with 13% of the hours available and 12% of the hours actually used. 
Mathematics represents 49% of the programed hours available and 
31% of the programed hours used this year. Sciences represent on- 
ly 14% of the hours available, but involved 31% of the hours used. 
Social science is 4.6% of the hours available, was 4.7% of the hours 
used. Other subject matter covers 11.4% of the hours available and 
only 4.7% of the hours used this year. 
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PROGRAMS by GRADE LEVEL 






25. Grade Level 




Percent of programs used 



Percent of programs available 
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PROGRAMS by GRADE LEVEL 

In a comparison of the number of programs usable at each grade 
level, 4% of the available programs are listed a$ appropriate for 
first grade level; 6% in the survey were used in ‘first grade. Four 
per cent are suitable at second grade level, 6% were used at that 
level. Eight per cent are available for third grade level, 7% were 
used at that level. Sixteen per cent of the programs available are 
available for each level in the fourth and fifth grade; only 8% were 
used in each of these grades. Eighteen per cent of the newer pro- 
grams available are aimed for sixth grade, only 7% were actually 
used in that grade this year. Twenty -four per cent of the programs 
to be available are for seventh grade use, 10% were used in seventh 
grade. Twenty -eight per cent will be available for eighth grade use, 
only 21% were thus used. Twenty-five per cent of the programs 
available are for ninth grade use, and here the trend reverses: 42% 
of the programs surveyed were used at ninth grade level. Sixteen 
per cent of programs available are 10th grade level, and 31% were 
used this year at the 10th grade level. Twenty-one per cent are 
available at the 11th grade level, 25% were used. Twenty -one per 
cent are available at the 12th grade level, 24% were used. 

This data indicates that the greatest degree of usage is now go- 
ing on in grades nine through 12, and that although somewhat less 
was done at eighth grade and below last year, the availability of 
materials is improving for these levels. 



PROGRAM LENGTH and COST 

The next comparison involves figures difficult to justify ac- 
curately but interesting even as a crude approximation. The median 
number of frames in the programs available in Programs, *62 is ap- 
proximately 2,000 frames. The modal number of frames is closer 
to 1,000, while the mean is over 3,000 frames. These three figures 
simply point out what shows graphically in Figure 6 ( p. xxi) of 
Programs, *62 , that the distribution of program length is highly 
skewed towards the few programs that are very long (over 10,000 
frames). 

Using the same figures in conjunction with the cost per frame 
data also supplied in Programs, *62 one can arrive at an average 
program cost which ranges between $2.00 and $57.00 (the upper 
limit is not finite, but for the purpose these figures will suffice). 

The median program will cost around $12.00 and the mean program 
(by length) would cost closer to $19.00. One might therefore say 
that a loose average cost of available programs is between $12.00 
and $19.00. 
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PROGRAM LENGTH and COST (continued) 



In terms of the data of the usage survey, these cost figures are 
really remarkably close. The largest single category, checked by 
37% of the respondents, paid between $10.00 and $15.00. In other 
words the cost of programs in the last year or two averages be- 
tween $10.00 and $15.00 and judging by presently available figures, 
that cost is likely to be maintained in the immediate future. 
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IV 

RESEARCH AND INFORMATION WANTED 

Responses to the question “What research would you like to 
see done?” indicate a wide diversity in the definitions of the term 
“research.” 

One group of comments requested the development of pro- 
grams, particularly at the elementary level. A second group of re- 
spondents are concerned with the evaluation of existing materials. 
Evaluations desired range from a simple stamp of quality assessment 
ment to a fairly sophisticated breakdown evaluating groups in terms 
of age, intelligence, and so forth. Another group is willing to 
evaluate programed materials themselves, but finds they do not 
have personnel or know-how. One comment specifically requests 
statistical evidence verifying publisher claims; a variant on this 
theme wants publisher data and research on every program devel- 
oped with the hope that this material will help increase quality, 
and a third comment speaks for itself, “We have examined quite a 
few sets of programed materials. We have been totally unwilling 
to seriously consider some of them because of the disregard ap- 
parently held for research and curriculum development in the field 
[by] the program writers.” A different, but familiar category of 
comments continues to demand the evaluation of programed mate- 
rials in comparison with conventional materials. This demand for 
“proof” of the innovation is a most logical and persistent one; there 
are many psychological factors to be faced in any new technique — 
and some of the most critical eyes in the world of colleagues, par- 
ents and boards of education will be focused on this matter of the 
end result: “Does it work?” “As well?” “ Demonstrably better?”. 

Somewhat related is the group who wants to know more about 
what program users do— to what extent they have used programs, 
in what manner, and myriad utilization details, asked, one sus- 
pects, to guide local introduction of these new materials. To a great 
extent it is this group, and certainly this purpose, for which the 
present report has been compiled. At least the beginnings of simple 
utilization data are here available in answer to the comments re- 
questing information on small school use, use in combination with 
texts, and even large scale field trials. 

Several users want information in greater depth than is yet 
available; a half dozen want research findings on the value of pro- 
gramed material in remedial instruction; almost as many request 
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RESEARCH AND INFORMATION WANTED 



research on the use of programs with especially talented students. 

A suggestive comment in this connection comes from a school sys- 
tem whose initial use of programed instruction was for the talented, 
but which is now planning to broaden its use to include a wider spec - 
trum of ability levels. One comment encompasses this whole area 
to ask for research on the effects of programed instruction upon 
the individualization of instruction while another pertinent com- 
ment asks for research on the problem of utilizing programs with 
whole class groups. 

There were many comments on the matter of program costs*, 
some of these recur again in this question of research that schools 
would like to see done. One such comment comes from a school 
system well involved in programing: “I see this problem in two 
parts. One, teachers trained to write programs (I cannot justify 
the use of a substantial amount of school funds for this purpose). I 

Two, teachers trained to become more sophisticated in the use of 
programed materials (for this I can justify use of school funds).” ' 

Other cost comments run from “No financial problems at this 
time,” through “Yes, if usage is extended,” “The cost of materials 
exceeds textbook budget,” “The expense of programs and machines 
is high and much material is not reusable,” “Moderate, it will re- 
quire budget adjustments,” “The initial investment is so great; 
some teachers would like to use programed material [completely], \ 

but finances prohibit [such costly] purchases.” Several users men- j 

tion the specific problem of clearing texts through a state approval ; 

source and this is both a curriculum and a financial limitation. One 
commenter estimates that the cost is going to be about four times 
that of textbooks. And a last financial comment, “Lack of funds 
available presents problems in securing material and equipment 
to carry out research... program budgets are exhausted to the limit.” 

This last comment must be weighed against the situations reported 
on the financing of programs where the great majority of the schools 
using programed materials are both currently operating and plan- 
ning to continue their use of programs within their regular budget. 

A final category of comments checked on the question of re- 
search desired is illustrated by what may be a fairly representa- 
tive situation. “Information on ways of utilizing and research being 
done is perhaps available but we need a person with time to read, 
report, and study it.” “A system such as ours is too large for us to , 

be able to spend the time necessary to study such materials as pro- 
gramed learning and our knowledge is sketchy, and yet we are too 
small to afford the personnel necessary to keep up with such devel- 
opments.” 
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RESEARCH AND INFORMATION WANTED 



All in all, while a great many of these comments are not di- 
rectly pertinent to the pure form of academic research, the need 
they show for additional information, guidance and evaluative help, 
is quite vivid, and the willingness to look at research reports is 
certainly highly appropriate to a young field like programed in- 
struction. The fact that the searching for information is not com- 
pletely uncritical is a matter of considerable encouragement for 
those involved in the research of program fundamentals and appli- 
cations. 



V 

SUMMARY AND DISCUSSION 



APPRECIATION 

In general surveys, a return of 20% is considered very reason 
able; of the nearly 15,000 school systems to whom these question- 
naires were sent, the return of over 3,J300 forms is a rather re- 
markable tribute to busy school administrators and teachers all 
across the country who gave time and thought to their return. 



SUMMARY : Users — Non-users 

The overall familiarity of even non-users with terms of the 
programed instructional field is an indication of the impact of this 
new field upon education. For both users and non-users profession- 
al publications play the dominant role in providing initial informa- ; 

tion about programed instruction. The utility of Lumsdaine and j 

Glaser's collection of articles for the NEA shows in their domi- i 

nance of the early literature in programing. 

The gathering of sample programs is the predominant informa- 
tional activity of both groups, although a surprising 34% of users \ 

are using consultants along with their programs. The users have 
also made more of an attempt to inform lay groups about pro- 
gramed instruction. 

Most schools using programs are at the “few student* trial 
stage and a few are using their programs as part of regular group 
instruction; «ome even incorporate research design and controls 
in more formal experimentation. Enrichment and remedial instruc- 
tion vie close’ / with regular instruction as purposes for which pro- 
grams have been used. Non-machine use of programs far exceeds 
machine usage at present. And the regular budget accounts for al- 
most 90% of both present and planned school financing of these ma- 
terials. 

Teachers and curriculum coordinators initiated most of the 
program use, but 50% of the responding users did not even answer 
the question as to whether teachers were provided time for using 
or developing programed materials; a few larger systems have j 

made some provision in this respect. j 

It apparently takes less than a year from initial contact to j 

program implementation. Reactions are generally favorable from j 

all groups: parents, teachers, administrators, boards of education j 

and students; and the using schools are strong in expressing a need j 

for additional information and research in the field. j 
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SUMMARY: Users — Non-users (continued) 



One generality may be made in connection with the’ size of 
school systems responding. The early users of programed instruc- 
tion are likely to be the larger school systems. 



SUMMARY: Individual Program Usage 

Mathematics programs are the most available and most used. 

While most programs are obtained from commerical sources, a 
respectable 19% have been locally programed, usually by a subject 
matter teacher. The teacher is also most often the director of any 
program project; most usage is presently on a limited trial basis. 

Program cost appears to be between 10 and 15 dollars per copy. 

Non machine formats predominate, and most usage is without 
supplementary materials. 

Present programs are being used mostly at the junior and 
senior high school levels, however, almost half of the programs are 
also reported used at or below sixth grade. The students involved 
were usually of average intelligence or above; for them the pro- 
grams were a regularly scheduled part of classroom instruction, 
with the teacher in an active role. 

Evaluations of program content were generous (almost half 
considered their program content * modern”) and while many re- 
spondents felt that it was too soon for overall evaluations, the 
responders clearly lean towards the favorable side. 

Program publishers would be well advised to note the variety 
of information respondents would like to see accompanying any 
new program: a statement of objectives, content classification, 
sample frames, and author -indicated criterion frames. 

Two breakdowns of the returns by size of system enrollment 
are presented at the end of Parts I and n. 

The programs used are compared in Part III with the programs 
available in September 1962. Mathematics still dominates, but less 
strongly in the proportion of programs becoming available. The 
number of hours of science material in use is second to mathemat- 
ics, and will apparently continue to be. While present usage is cen- 
tered at the high school levels, both interest and availability are 
spreading rapidly into the elementary level. 

A further key is offered to program producers and researchers 
in Part IV. While many users want programs graded and stamped, | 

another group wants information and assistance which will enable 
them to form their own evaluations. 

j 

3 

1 

5 



| 



iitKiii mmmmM 



iaiii 






DISCUSSION 



Some areas mentioned briefly in preceding sections are here 
brought together for comparison and more complete discussion. 

MACHINE VS. TEXT 

One hears much discussion and opinion on the question of 
whether or not hardware is necessary in the use of programed in- 
structional materials. This survey contributes certain pertinent 
facts: 

a) The non-using schools are undecided . 

Seventy -three per cent of non-using schools are at present 
undecided as to the form in which they will use programed instruc- 
tion in the future. The number of non-using schools who plan to use 
programs alone, (i.e., without machines) is small; so is the number 
who plan to use programs in machines (Fig. 5). 

b) The large majority of schools using programed materials 
are not using machines . 

While the specific question of machine vs. text was not asked 
of program users, information they furnished on form of program 
presentation provides a basis for comparison. Horizontal texts were 
used by 28% of the program users, vertical texts by 38%, and ma- 
chine programs without machines were used by 7%. These “paper” 
programs add up to 73% of the individual program usage. In con- 
trast, teaching machines were used by 12% of the total. Of the ad- 
ditional 7% who responded with multiple checks on this question, 

3% used combinations of other formats (horizontal, vertical, etc.) 
with machine presentation and 4% used non-machine combinations. 
(The 2% checking auditory presentation would presumably also need 
some variety of hardware). 

So, while the term “teaching machine” has both caught the fancy 
of the press and frightened certain elements of the public, the 
schools have obviously not been stampeded into using the hardware. 
Instead of centering their attention on teaching machines, the schools 
are interested in and using the programs themselves. 



PURPOSES and INTENTS 

Some interesting variations in plans for program use and de- 
clared intents for which programs were actually used are compared 
in Table I. 
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PURPOSES and INTENTS (continued) 



Table I 



Two Dimensions of Program Usage-Purposes and Intents 



Remedial 

Regular Instruction 
Enrichment 



Non-Users Users Individual Program Usage Reports 
(Planned) (Admin.) (Intent) (For Mental Ability Level) 
29% 55% 23% Below Ave. 35% 

20% 68% 74% Average 60% 

34% 60% 24% Above Ave. 58% 



Trial Basis 
Integral Curriculum 
w/Research Design 



57% 


54% 


43% 


27% 


23% 


21% 



Non-using school systems were asked their plans for program 
usage. Using school systems (responses usually filled out at the 
higher administrative level) were asked both their plans for similar 
materials in the future and their degree of present involvement in 

research. Individual Program Usage sheets contained three related 
items: 

1) A statement of overall purpose (the experimental involve- 
ment again, “On a limited trial basis,” “As an integral part 
of curriculum,” or “With controls and research design”). 

2) Number of students using the program in each mental abil- 
ity level and, 

3) Number of students for whom the usage was with remedial, 
regular or enrichment intent. 



The non-users do not plan to use programed materials primar- 
ily for regular instruction. Using systems, on the other hand, indi- 
cate that on the basis of their experience with programs, 68% plan 
to use similar materials for regular instruction. And the individual 
prograin user makes it quite clear that the primary intent of his 
use of programs was for regular instruction. Whether these figures 
may be interpreted consecutively is moot; programed instruction 
may have an initial appeal for its special uses, either remedial or 
enrichment, but with actual experience the appreciation of its pos- 
sibilities for regular instruction would seem to grow perceptibly. 
Even the mental ability level of the largest proportion of student 
usage favors the average. 

Users at the administrative and operating levels concur in 
their overall purpose of program usage on a “limited trial basis” 
and “with controls and research design.” A minor discrepancy 
occurred in the proportions who indicated that they were using 
programs as an integral part of the curriculum (for regular in- 
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PURPOSES and INTENTS (continued) 



struction). Forty-three per cent of the administrators checked 
their present use as “for regular instruction* while only 27% of the 
individual program users checked (as a statement of overall pur- 
pose)— “an integral part of the curriculum.” 

Whether this is due to a lack of communication between these 
two levels is hard to say. Certainly those closer to the program 
usage level would appear to be aware of the trial or experimental 
nature of the operation. From the point of view of higher adminis- 
tration, it is pretty literally true that a great many students were 
getting regular instruction, even though they were involved in trial 
or experimental program usage. A further explanation of this dis- 
crepancy may come from the fact that the administrators more 
often checked multiple purposes (their percentages total well above 
100), whereas, at the program using level respondents apparently 
had a more specific (and possibly more realistic) knowledge of 
their activity and therefore checked only one purpose (their per- 
centages total much more closely to 100). 



INNOVATION in EDUCATION -A Case Study 

Dr. Henry Brickell’s study of instructional change in New York 
Stated covers so many of the problems in innovation that it may be 
useful to review some of the points he makes (italics, below) and 
check them against the reports of users of programed instruction. 

The rate of change in foreign language, mathematics and sci- 
ence courses tripled in the 15 months after Sputnik was sent 
around the world. In this survey of programed instruction, only 
mathematics seems to be maintaining this leading role (Fig. 23.). 

The non-public schools do not match the rate of change of the 
public schools. The few non -public schools sampled in the present 
survey would sustain this impression. This pattern has also ap- 
peared in the introduction of new curricula. 

While initial interest in programed instruction has been largely 
expressed by the high schools, the public elementary levels are 
already using some of the “higher level” programs and as more 
materials come into view their interest seems to be rising rapidly. 

Most of the attention paid to course changes in public schools 
resulted in new programs for above average students in junior and 
senior high schools. 



14 Brickell, Henry M. Organizing New York State for Educational 
Change, State Education Department, Albany, New York, 

1961. 
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INNOVATION In EDUCATION— A Case Study (continued) 

In the present usage survey, the average student seems to be 
getting a better break. Sixty percent of current usage is with 
average students (see Table I and accompanying discussion, above). 

Dr. Brickell’s discussion on the role of the public, the board 
of education, and the school administration seems well supported 
in programed instruction usage. Usage occurs most easily when 
parents and board are not actually opposed (see Fig. 11.), and when 
the administration gives not only attention but shows some activity 
in favor of the innovation. 

The classroom teacher can make only three types of instruc- 
tional change in the absence of administrative initiative: change in 
classroom practice, relocation of curriculum content already in 
existence, and the introduction of a single special course at high 
school level. The many teachers who have been initiators in pro- 
gramed instruction may have done so under the first category, 
change in classroom practice. 

Other faculty members ordinarily begin to prefer new methods 
within four months to a year after a novel program has been intro- 
duced, regardless of their very early reaction. Certainly this period 
had passed for the using respondents (Fig. 9.) and the teacher reac- 
tions support Brickell’s observation without further comment. 

(Fig. 11.) 

Brickell reports finding some evidence that the greater the 
time lag between learning that a new program was to be introduced 
and actually beginning it, the greater the opportunity for teachers 
to become anxious. This hypothesis is only slightly supported by 
the present data on programed instruction. Most of the programs 
that took over a year to get into action report a teacher reaction of 
“favorable,” some are “neutral,” and a iew “enthusiastic.” How- 
ever, and clearly in support of Dr. Brickell, all of the systems re- 
porting an “opposed” reaction from teachers did in fact take a year 
or longer from initial contact to trial use, and most of them took 
six or more additional months to get programs into the classroom. 

The real source of rigidity in an educational program is not the 
written guide or textbook, but is the teacher who knows no more 
about the subject than is contained in that guide or book. As one of 
the newer instructional approaches which offers greater flexibility, 
programed instruction has already run up against this problem with 
the teacher, who now has material with which the student must work 
on an individual basis. Understandably, many a teacher feels con- 
siderably more lost than when he worked with students on a familiar 
group basis. The quality of teacher flexibility involves more than 
academic knowledge, and nothing is going to dramatize the need 
for teacher ingenuity and flexilibity any more forcefully than pro- 
gramed instruction 
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INNOVATION in EDUCATION— A Case Study (continued) 

Instructional innovations are almost always evaluated by ob- 
serving the reactions of the students while they are receiving the 
new instruction. Certainly local school systems cannot be expected 
to use anything other than observational evidence. Conversely, 
teachers who are interested in observing and evaluating new tech- 
niques such as programed instruction should be given the opportun- 
ity to talk with the students. Brickell says that over many months of 
talking with large groups of teachers there is no doubt whatsoever 
but what the teachers quote about student behavior is the single fac- 
tor which carries the greatest impact. 

In a check of evaluations of “overall efficiency 1 ’ of programed 
instruction (Fig. 20.) as compared with reported student reaction 
(Fig. 11.), of the schools judging program usage as “extremely 
worthwhile,” 24 reported a student reaction of “enthusiastic,” 
eight “favorable,” and only one as low. as “neutral.” On the other 
hand, none of the schools judging program efficiency as “adequate” 
or less reported a student reaction higher than “favorable.” 

Instructional procedures which students react to with interest 
or enthusiasm are ordinarily judged to be successful. The alter- 
nate for this is no more available for programed instruction than 
it has been for most other innovations. “When new instructional 
programs are invented adequate methods for evaluating them are 
not invented simultaneously.” When researchers object to the 
schools saying, “but this program works,” they should remember 
Brickell’ s statement that a school using a new program has the 
choice of either measuring it with the “wrong” yardstick or making 
a rough subjective estimate by observing student response. “Nei- 
ther alternative is adequate for a [wide spread] program of accel- 
erated change. Something better will have to be designed” (p. 34). 

Another point connected with user reaction as the sole method 
of evaluation has to do with the familiar phenomenon that almost 
everything new works better. The attention, encouragement, and 
recognition given to teachers by people outside the classroom dur- 
ing the intvoduction of new pvogyams aye among the styoyigest 
causes for this success. For Brickell this points out the necessity 
for using comparison groups which use an old method but which 
are supplied with the same ingredients which energize the new 
method, teacher enthusiasm, attention, visitors, and so forth. 

The professional associations are the supreme communica- 
tors in the profession. . . the persuasive effect of printed material 
is relatively slight when compared to the informal contact among 
colleagues. The present respondents, both the user and the non- 
user, report their initial contact as far greater through publications 
than through colleagues, although colleagues were the second source. 
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INNOVATION in EDUCATION -A Case Study (continued) 

“Initial contact,” of course, is not necessarily “persuasion.” This 
study actually parallels Brickellk findings in that professional asso- 
ciations, are the most frequent source of professional literature. 

Commercial organizations such as text book publishers are 
extremely powerful. On the other hand, once they begin to market 
a product they serve as powerful inhibitors of change because they 
want wide distribution and repeat sales of the same product. In no 
field within recent recall is this truer than in programed instruc - 
tion where teaching machines were often the teacher’s first contact 
with this new potential method. The second point of concern in 
commercial developments is that machines tend to be limiting 
factors in the design and use of programed materials, having in 
most cases, neither the flexibility desired nor the arrangements 
for many modalities which will eventually be needed. Says 
Brickell, “It is inconceivable that commercial organizations will 
stand aside over the next decade, let us say, while the few people in 
the profession who have the inclination, skill, time, and resources 
proceed to write new programs, test them in pilot applications, 
revise the materials, evaluate them thoroughly through extensive 
field testing in schools, and then somehow arrange for volume 
production and volume distribution. “The commercial concerns 
have already moved heavily into the area... they will undoubtedly 
play the dominant role” (p. 61). At present 24% of program users I 

acknowledge directly that their choice of program was based on a 
publisher statement. The most formidable block to instrucational 
improvement today is that education. . .fails to distinguish the ; 

three phases of change [which have distinguished science and in- 
dustry]: The design of the innovation is properly apart from reality, j 

the evaluation of innovation depends upon close surveillance, and 
the dissemination of a new approach must be through circumstances ; 

as “normal” and “real” as possible. j 

The design of programed instruction certainly blossomed in j 

the laboratories of psychology as the same principles in actual j 

classroom situations over many years did not. The present danger j 

in programed instruction is in its adoption too thoroughly too early; j 

more basic laboratory research is still needed to amplify our com- j 

prehension of the principles behind successful programing. 

While field testing of programed materials is under way, the \ 

proportion under carefully controlled conditions is still low (20% j 

at most optimistic estimates). The need is for long range and probably ex- 
ternal guidance for large scale and carefully controlled local experimen - 
tal usage. Local interest in “ improving the product” needs to be supple- 
mented by representatives of more general knowledge and basic relation- j 

ships involved. $ 
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INNOVATION in EDUCATION -A Case Study (continued) 
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Dissemination of programed instructional techniques is per- 
haps the area most closely resembling Brickell’s observations and 
suggestions. Few schools are adopting programs without seeing 
them in use in situations which to them are normal and real. 

Our greatest barrier in... improving education. . . will be our 
loyalty to the inadequate organization and arrangements which 
exist today . 

Perhaps the greatest problem in the future use of programed 
materials is only indirectly brought out by the questionnaires used 
in this survey, although some comments mention it specifically. 
When a student has completed a programed unit, the knowledge, 
ingenuity and diplomacy of the teacher is regularly tapped in 
guiding the student to his next most appropriate activity. Once 
the lock step of “uniform appointments” is broken, the guiding 
and advising role of the teacher may far outweigh the straight 
“lecture -instructional” role of tradition. This change must be 
understood and accepted by the teacher, and the system which 
supports him. 



CONCLUSION 

The general picture of program usage, despite all incomplete 
and difficult problems, emerges as encouraging for those schools 
who wish to experiment. While the gains remain hard to quantify, 
the full “Hawthorne” effect of attention is completely available, 
and the response of using systems shows that almost all parties 
involved react favorably to the use of programed materials. 

While basic research and program evaluation remain general 
problems, one of the most acute practical problems likely to be 
faced by users of programed materials is that of individual student 
rate of progress. And while the solutions are not yet facile, nor 
immediately likely to be, the problem of increased individual 
student activity should find many willing grapplers in education. It 
is a problem almost uniquely forced by the use of programed 
instructional materials. 
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N = 1671 



Tho Centor for Programed Instruction, 365 West End Ave., N.Y.C. 24, N.Y. 

U.S. OFFICE OF EDUCATION SURVEY 

This ihm to bo filled out ONLY by tbo»o school superintends ts having LITTLE of NO EXPERIENCE with PROGRAMED INSTRUCTION 



Is Art you fomlllof with any of that* farm*: Programed Instruction, Too chin g Machine, Programed Looming, Automotlc Teaching, ate# 



yes, no 

84 % 



05 

O 



2. Hove you ever rood on artlclo on progromod instruction? 



Ves no 

90 % 



3* Hove you ever soon o teochfng mochlne? 
yti no 

73 % 



• II "yes" to any of the above questions, please continue to fill out this shoot. 

- II "no" to oil thro# questions, pleoto fill out tho llnol section o! this one sheet and return It to the oddress obove. 

4. Pleoso chert your initial source of Information on programed moterlols. 

Newspopers Professional Loy Members Other 

ond Mogoilnet Publications Colleague of Community Soletmen (Specify) 

32 % 76 % 12 % 1 % 10 % 13 % 



Non User" Questionnaire - Showing Response Percentages 






j ig ii, iil 4PJ ^l!Hi J|>|1 



5. Of which of the following hove you rood ot Uott two issues? 












Automoted 


Programed 


J. of Educ’l 


Other with Programing 


A-V Instruction 


AID 


Teoching Bull. 


Instruction 


Psychology 


artlcles(speclfy) 


35% 


6% 


6% 


24% 


15% 


15% 


6. Which books have you rood? 












Teoching 












Mochlnes ond 


Exploiting 


Teoching 






Auto mo ted 


Programed 


Teaching 


by 


Programed 


Other 


Toochtng 


Looming 


Mochlnes 


Mochine 


Primer 


(specify) 




(Lumsdolne 










(Golontor) 


and Glaser) 


(Cramm) 


(Stolurow) 


(Morkle) 




at . 3% 


15% 


0% 


3% 


3% 


2% 


t-» 












7. Hovo you examined 


o program? 










yos 


no 


(omit? 








44% 


46% 


10% 








8. Which of the following slept regording Progromed Instruction hove been token In your system? 






Colled: 




Set up o 


Appointed 


Sent Stoff to 




A School System 


School 


Plonn'ng 


A Program 


A Programing 


Utilised 


Conference 


Meeting 


Group 


Director 


Workshop 


Con sultan ts 


9% 


ii% 


6% 


\ % 


13% 


n% 




APPENDIX A - "Non User" Questionnaire (continued) 
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o 

ERIC 



8. Which of the , low In* .up* fepordlnQ Progromed In.troctlon ho»o boon tokon In your .y.tom? (confd) 



Obtolnod 


Tokon any stop! 










Program 


To Inform Lay 




Othar 






SompUi 


Group! (parantt, ate.) 




(opacify) 






35% 


8% 




10% 






9, Which of ihoio mm itopi do you plon to toko In your »y»to«n? 








Coll: 




Sat up a 


Appoint a 


Sand Stoff to 




A School Syitom 


School 


Planning 


Program 


A Programing 


Utilize 


Con for one a 


Moating 


Group 


Director 


Worfcohop 


Con*i’'tonti 


8% 


0% 


19% 


2% 


12% 


18% 


05 Obtoin 


Takr ^tPps to 










^ Program 


Inform 1 ay Group • 




Other 






Samploo 


(par onto, ate.) 




(opeclfy) 






35% 


14% 




5% 


■ - 




10o Do you Inland to 


uia program! 


Alone 


In Machinoo 


Undarldad 


( omi t) 






7% 


5% 


70% 


10% 


11. Do you inland to 


ubo programed malarial i for 














Remedial 


Ragulor 










Work 


Inotrurltoo 


Enrichment? 


( some 






29% 


20% 


34% 


multipl 

phnrkcl 









APPENDIX A - "Non User" Questionnaire (continued) 



I Tin jjliii 11 ' ' 



in nww 




l- <. 

: 




: 



> 

12* Your funds will (probably) coma from 

Regular Extra State Federal 

School Budget Locol Funds Funds Funds 

68% 3% 2% 6% 

> 

Nam* ond title of ptrson filling out quttilonnalrt , 



Othtr 

(iptclfy) 

2 % 



TJ 

M 

2 

O 



Name of School System 

13. Total Pupil Enrollment 


0-499 


500-999 


1000-4999 


5000 ♦ 




20% 


18% 


37% 


13% 


14. Type of System 


Rural 


Urban 


Suburban 


( other) 




35% 


15% 


o 

CM 


5% 



This questionnaire has been □ easier Q same as Q harder than the many others you receive* 



Comment: 



S 

O 

3 



a 



CO 

<D 

n 



£> 

p 

a> 

:o 

•4. 
f J* 

o 

3 

P 

j-j. 

10 



o 

o 

a, 

5 : 

P 

a> 

a 



ERiC 












N = 209 



Center lor Programed instruction, 365 West End Ave., N.Y.C. 24, N.Y. 

OFFICE OF EDUCATION SURYEY 

Thl» sheet lo be titled out ONLY by those school superintendents whose SYSTEM S ARE USING PROGRAMED INST R UCTI O NAL MATE RI AL S. 



KLC 



o> 



1. Who* wos your Initial i 


seurce of information on 


programed Instruction? 








Newspapers 


Prole sslonal 




Lay Members 




Other 


/ Amt 4 1 


or Moootlnot 


Publication 


Colleagues 


of Community 


Salesmen 


(Specify) 


1 om) t ) 


16% 


72% 


23".; 


1 V, 


8.6% 


16% 


6% 


2. 01 which of the following have you rood at leost two issues? 














Automated 




Journal of 






A-V 




Toothing 


Programed 


F.ducotionol 


Other 




Instruction 


AID 


Bulletin 


Instruction 


Piyrhology 


(Specify 




5 I'm 


20*:., 


25" 


60% 


33% 


7% 


12% 


3. At you probably know, 


, teaching machines are 


being sold from door to door. How mony porents do 


i you know of who hove bought such m 


oterials? 


Mono 


A handful 


A considerable 


number (estimate? 


) 






4% 


73% 


23% 










4. Wlilch books ha v« you 


read? 














Teaching 








Olhet 






Mm bines ond 


E xplolning 






(Sprrlly) 






f 'rtigrmned 


Teaching 


T eac hinq 








Automated 


Learning 


Machines nnd 


by 


Programed 






T eurhing 


{Lumsdolu# 


Programing 


Mochtne 


Primer 


Author; 




(Golonter, 


ond Glaser) 


(Crttmm) 


(Stolurow) 


(Morkie) 






14% 


54% 


19% 


7% 


19% 


2% 


35% 


V Which of the following steps regarding Prngrnrr 


ted Instruction hnve been token in your system? 








School System 






Appointed o 


Stolf Sent to 






Conference 


School Meeting 


Planning Grojfc 


Program 


A Programing 


Consultants 




Called 


Called 


Set Uo 


Director 


Wbrkshap 


Utilized 




22% 


c. O/n 


32% 


Q% 

y,n 


33% 


34% 




Program 


"Slops Token 












Sample? 


To Inform Lay 


Other 










Obtained 


Groups (Parents, etc.) 


(specify) 


omit 


_ _____ „ 






82 % 


47% 


10% 


2% 












i. 

















APPENDIX A 

"User" Questionnaire - Showing Response Percentages 





6. Which of Hits* steps do you plan to tohs In your system? 



7. Whot use of progroms hove you mode? 

Tried out on Tried out on 

Indlvfduol Groups of 

PupJL? .Pufilil 

46 % 80 % 

8. Which use of progroms do you plon? 

To try out on * To try out on 

Indlvlduol Groups of 

P upl Is P upil s 

31 % " 52 % 



Developed 

Own 

Progromf s) 

17% 



To Develop 
Own 

Program's' 

17 % 



05 

C71 



9. Are touchers using or developing progroms doing so 
On Their On Releosed In Pold 

Own Time Time Overtime 

2/o 



44% 1 5% 

10. First progrom use In your system Is traceable to: 



A T each er 

22 % ~ 



A Curriculum 
Coordinator 

21% 



An A-V 

Director 

5 0 / 

/n 



Appoint a 



.School System 


Will Coll 


Set up o 


Progrom 


Conleren ce 


School Maatlna 


Plonnlng Group 


Director 


8% 


6% 


17% 


Qn/ 

O/o 


Obtolri 


Tdfca Slapi ld 






Progrom 


Inform Loy Groups 


Othar 


{omi t ) 


Sctmoltl 


(Eofenll*_etCAl 


(>P.*cJ.fjt} 


43% 


32% 


5% 


31% 



In Pold 
Vocotions 

4% 



A Pilnclool 

20 % 



Sand Stoll to 
A Programing 
Work shop 

22 % 



Utlllto 

Consu ltant* 

27 % 



Tested 


Exparlmantol 


Other 


.11 
— C< 

|CO 

w* 


Hvpothaaai 

14% 


(specify) 

2% 


To Test 


Experimental 


Other 




Hypotheses 


_(*£•?"*) 


.. — 


™ 18% " 


3% 


28% — 



"school" or 
" rwju 1 ;ir M t i mo 
4% 



Other 

Admin, 

(specify ) 

32 % 

(stipl. most 
o f ton) 



( omi t ) 

1% 



52% 



Interested. 

toymen 

3% 



Any Other 
(specify) 

Q’V 



( omi t ) 

8 % 




APPENDIX A - "User" Questionnaire (continued) 



11. How wo* your u*e of projrumed .' oMrrioJ financed* 
c xtro City 



Regulor 

School 

Budget 

90% 



Or Locol 
Fund* 
2 \ 



Slot* 

Fund* 

2-'. 



12. How do you pi 


to finance future progrom ut»e? 




Regular 


Extra City 




School 


Or Locol 


Stote 


Budget 


Fund* 


Funds 


90 


3 


3 


13. Your progrom* 


were used; 




On o Limited 




For Regular 


Trial 0o*l* 




Instruction 


57 




.43 



Federol 

Fund* 

4 % 



Federol 

Fund* 



Publuhsr 

7 11/ 
u 



Publisher 

5 



With Controls 
And Research Deslan 

23 



Urhorn'y'oof first contact with Progromed In.tructlon to octuol u.og. (ovon on on .xp.rtaMtal ko»U)s 

Imm.dlattly 1-2 mot. 3-6 mo». About 1 yi. Ow l yf* 

1 10 34 35 18 



15. from *uch first experimental use of programed moterlol to octuol clossroom Instruction; 

. mm.dloi.lv 1-2 mot. U .mo.. About 1 yt. Oy.r » y.ot 

g 20 6 23 19 7_ 

16. On th. bad. of your .xp.rl.nc. with Programod Inttructlon, do you plan to uto tlmllor motariol* In luturo lor: 



Remedlol 
Work 

55 



Regular 
Instruction 

~ 60 
Enthuslostlc 



Enrichment 

60 



Not ot 
All 

1 



.FovofobU 



Noutrol 



17. 


Reoctlon from Teachers 


22 


52 


13 


IS. 


Admin. 


27 


53 


9 


19. 


Porents 


12 


37 


23 


20. 


Bd.of Ed. 


12 


45 


19 


21. 


Students 


24 


40 


11 


22. 


Other 

(specify) 


0.5 


0.5 


0 



ORROfpd 

3 
1 
2 

0.5 

4 



Othor 
(too cl hr) 

9% 



Othor 

(specify) 

5 



Strongly Oppot.d 

0.5 



0.5 

0.5 



(Omit) 

2 % 



6 

3 



11 







APPENDIX A - "User" Questionnaire (continued) 



23. Do you fool thot poopto In your tyttom oro odequoteiy proporod to ovaluoto 
Non-pregramed Instructional matorlolt? Progromod moterlols ? 

ne *» (Omit) (Omit) 

79 14- 54 8 35 12 

Do you anticipate more than usuot ft ngncloLptob terns In futuro uto of Programs? 
yos no (OtTli t) Comment: 

21 70 9 16% 

25* What rosoorch would you like to seedohe? (use a separoio shoot If you coro to) 

26. Which of tho following oro you not getting enough Informotlon about? 

Other School Ways of Proaroms Progrom Reseorch 

JJlftfle. Utilising AvoTloblo Constructing Being Dong 

41 44 5i 32 54 

If you have octualty mode use of one or more programs — please fill out o yellow sheet for each program used. (This may, and *.*:Saps should, 
be done by the person directly In charge of each program.) Additional yellow sheets ore ovoilobleupon request. 

Name and title of person filling out this questionnaire 



Name of School System 


27. Total Pupil Enrollment 


0=421. 


500—999 


1000-4999 




jflQILt 




17 


li 


38 




32 


28. Type of System 


Rurol 


Suburbon 


Urban 


(Other) 






29 


35 


26 


3 





This questionnaire hot been Q easier Q some os £] harder thon the many others you receive 

Comments (15%) 54% 33%. 4% 



•tttaso 




> 

M 

z! 

O 

R 

> 



d 

03 

rt> 

"i 



<0 

% 

03 



P 



"J 

(!) 



O 

O 

3 



g 

( D 
tt* 









N = 282 



The Center for Programed Instruction, 365 West End Ave., N.Y. 24, N.Y. 



u.s. 


OFFICE OF 


EDUCATION 


SURVEY 


- INDIVIDUAL 


PROGRAM USAGE 


SHEET 




CPI 1s conducting this survey for the Office of Education, U.S. Dept, of Health, Education and Welfare, to determine current use of progromed materials. 
The Information will be published In an onnototed blbllogrophy- to be of ossistonce to schools In deciding on, or choosing between, programs. 

(Please supply a separate sheet of Information for eech program; extra yellow sheets will be sent upon request). 


1. Subject Area 

.Math 
61 */, 


.Scl.nc. 

3 


Foreign 

Language, 

4 


English 

21 


Sp.lllng 

4 


Social 

Studlgs 

3 


Oth.r 

.(specify) 

3 


(Omit) 
2 X 


2. Source 
Amer. 
Institute 

Research' 

0 


Center 

Programed 

.Instruction. 

1 


Encyclopedia 

Brltannlca 

Films 

47 


Groll.r 

..(TMI) 

10 


Harcourt 

..Brace 

17 


McGrow 

HIIL 

0.4 


Science 

Research 

^Associates 

2 


' 4 P •. ' 


US Industries 
(Doubleday) 

1 0.4 


Locotty 

Programed 

16 


Other 

(tpielfy) 

3 l 












'3-3 










1 Vr . m 


6 \ 












3. Numbr o< Coal.. Us»d 

SI 18 'ib 


24-49. 

27 


5Q-99 

16 


]Qft-2?9 

15 


jm* 

5 




1 



4. Copy Cost 
£re«._ 

! 8 



_Und.cJ,PD 

3 



.L0Q-2,?JL 

17 



3.00^4,99. 

5 



_5.00-9.99 

5 



J 0,03=15,00. 
37 



J0x*rJ5.Ofl(Mult) 

9 4 



14 



5- If toewlly programed: Nome of individual(s) constructing progrom: 

Pro gramed* Title 
Subject 

hotter Curriculum Audio-Visual Other i T 

I<W£h« C oo^din gtor P ent. V, . , (ip eciivLK Malt) 

I 14 1 1 3 1 



OMiiHfi 



mmmm 



ittatliiiiM 



APPENDIX A 

"Individual Program Usage" Questionnaire - Showing Response Percentages 






Subject 




Audio-Visual 

Dept. 












Matter 

3 m #1 s»n ef 


Curriculum 
r nnrriinatnr 


(uucifylU Mult) L = 






40 


j >3 


9 


i 


16 3 








7. Statenfa^t of Overall Purpose 
On a Limited As an Integral 

T-t _ 1 fl ^ m t m D«»t aI f* (l»Ul«lnm 


With Controls & 
Research Design 


Other 

Comment: _ - 








i riai ousts t ir ._ 

1 54 


27 


21 




5 






4 


8. Farm at Presentation 






Machine Program 
Without Machine . 








Cnrits 


Harix&j'tcl 

Text 


Vertical 

Text 


In Teaching 
Machine 


Audi Of J ^ 


ffliJfM,. 1 1.) 




1 o 


28 


38 


12 


7 


2 


1 7 


4 


9. SuoDlementarv Materials Simultaneously U*ed with Programs 




Overhead 

Projector^ 


Tape 

Recorder — _ 






Regular 

Tam# 


Moving 

Plrfur* 


Fhm 
Strife . 


Slides 


Phonoaraph 




1 43 


9 


10 


4 


6 


5 


3 




o> 

50 Other (aixcltv) 
1 8 














Omit) 

45 


Number at prudent* 
Using Program 


0-5 


6-20 


2I-* 


41 -IOC 


101-30® 


300t 


94 


10* in each Grade 




1.1 


0.4 


0.7 




0.4 


11. 


2nd 2.8 


1.4 


1.4 


0.4 






94 


12. 


3,d 0.4 


2.5 


1.1 


0.7 




0^4 


93 


13. 


4th 2.1 


0.4 


3.2 


1 ,* 


1.1 




92 


14. 


5,h 2.5 


0.7 


2.8 


1.1 


0.7 




92 


15. 


6,h 0.7 


1.1 


4.3 


0.7 




0.4 


93 


16. 


7 ‘ h 4.6 


1.8 


1.1 


1.8 




0.7 


90 


17. 


8,h 5.3 


3.6 


5.3 


3.6 


2 v ?> 


0.7 


79 


18. 


’*^10.3 


6.0 


10.3 


8.9 


6.0 




58 
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Individual Program Usage" Questionnaire (continued) 



~v.-nrv..r,~-T J«U J^ 1 - t..UUlJLL J W Wf^ 



Number of Students 

Using Progrom (Cont*d) 0-5 



19. 


10th 1 1 .0 ?• 


20. 


11* ’ » 


21. 


12* 9.2 



-4 

O 



22. In each Mental Ability Level 1 


23. 


Above Average J9 


24. 


A vs rags H 


25. 


Bslow Average? 1 1 


26. In each In class 


12 




location 


27. 


In study hall 


10 


28. 


At home 


10 




with this Infant 


10 


29. 


Rsmsdlal Instruction 


30. 


Rsgular instruction 


13 


31. 


Enrichment 


11 




and with Ttachtr 




32. As active supplemen 


9 


33. 


Available foe 


12 




questions 




34. 


As proctor anly 


5 



O 



Omit) 



6.20 


21—40 


41.100 


101-300 


300 ♦ 


% 


7.4 


5.0 


3.9 


3.9 


0.7 


68 


6.0 


4.6 


3.2 


1.4 




75 


B.2 


4.3 


2.8 






76 


□ Oth.r, 


»p*elfy) 










15 


12 


9 


3 


0.4 


42 


14 


15 


11 


6 


1 


40 


11 


9 


3 


2 


0.4 


65 


TT 


2 


18 


15 


2 


66 


4 


2 


2 


1 




81 


4 


4 


4 


1 




78 


6 


4 


3 


2 




77 


10 


19 


17 


13 


1 


26 


7 


3 


3 






76 


10 


17 


12 


10 


1 


40 


9 


8 


7 


4 


0.4 


59 


1 


1 


2 


2 


0.4 


89 
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APPENDIX A - ’’Individual Program Usage" Questionnaire (continued) 








Amount of School 


Schedule Devoted to Progromed Instruction 


»° )& hrr 
4 
z 
7 


fo 9_hrs. ^ 

3 

_2_ 

10 


to 3 hr* 


. over 3 hn. 




35. Hours/Day 

36. Doy s/Week 


jt. 

25 

0 

2 


tojju. 

56 

1 

5 


1 

jt— 

19 


1 

— SL 
46 


9 

11 


37. Scheduled Weeks 


1 at Imj 

3 


2-3 . 
4 


4-6 

10 


7 -ii 

10 


16-32. 

25 


Oxat32 

35 


14 


38. In choosing this program, oia you use 
The only on* 

AuH 1 lit klft 


Publisher’s 

Statement 


Curriculum 
Coordinator _ 


Subject MottPr 
Consultant 


Subject Mottec 
TeachtL 


Other . 
(specify! 


90 




1 11 


24 


23 


16 


42 


12 


Evaluation: 
















39. Content 
Dark 
Anil 


Somswhot 
Obsolete . 


Reasonobty 
Uo-to-Dale_^ 


M ojem - 


Too 

Advan«td 








0 


6 


37 


48 


0 








40. To measure what the students loomed from the us* of this pro 
Ct Psikllth*v £unnll*d T«»rhsr Construct*! 


jpam, did you use 


a test that was 
Other (seed hr) 








37 


33 


40 




7 






11 



41. Overall Efficiency 

UitUlA. 

0 6 



Adequate 

30 29 



Extremely 

WttfihwhlU. 

18 



15 



42. Helpful Informorlon about o new progrom would Include 
A Statement Some kind of Sample 

of Oblectlvee Content Classification Fromes 

44 53 39 



Author Indlcoted Other 

Criterion Frome s (specify ) 

21 10 



Commtnvi! 

Nam* of Parson filling out this questionnaire 

43. Title 23^Subject Matter Teacher 



16£urrlculum Coordinator Dept. 



Jj^OAer (specify) 



> 

*0 

H 

a 

d 

f— i 

X 

> 

1 



P 

a 



a 

c 

p 



•TJ 

(j 

OQ 



P 

3 

cj 

Cm 

p 

OQ 



.<0 

3 



(D 

CA 




Name of School System — — - , - Z 

44, Total pupil enrollment Q 0-49.9 l6 & □ 500-9991 127,' □ 1000-499^35£ H5000 *34 % 

1 2 s 4 

45. Type of System □ R«rol'f on % DSuburbon g'^' DUrbon 262 C*h<v 5% 

1 2 9 



O 

o 



3 



U 

CD 

a 
















^ ,J; 





V 


t 











APPENDIX B 







TABLE II REACTIONSt 



ENROLLMENT 


Enthusiastic 


Favorable Neutral 


Opposed 


Strongly 

Opposed 




1 


2 


3 


4 


5 








Teachers 






0-499 


13 


14 


5 






500-999 


5 


14 


3 






1,000-4,999 


15 


47 


11 


1 


1 


5,000 + 


12 


32 


8 


_6_ 




Totals 


45 


110 


27 


7 


1 






Administrators 






0-499 


17 


16 


1 






500-999 


6 


15 


1 






1,000-4,999 


20 


42 


& 


1 




5,000 + 


11 


37 


9 


J_ 




Totals 


56 


111 


19 


2 










Parents 






0-499 


8 


12 


9 






500-999 


2 


7 


10 






1,000-4,999 


11 


30 


19 


1 




5,000 + 


4 


26 


11 


_4_ 




Totals 


25 


77 


49 


5 








Board of Education 




0-499 


10 


16 


4 






500-999 


1 


13 


6 






1,000-4,999 


10 


41 


13 






5,000 + 


4 


21 


17 






Totals 


25 


94 


40 


1 










Students 






0-499 


17 


13 








500-999 


3 


13 


6 






1^000-4,999 


20 


43 


10 






5,000 + 


11_ 


29 


7 


8_ 




Totals 


51 


100 


23 


8 


1 



tEntrie* are ft of schools checking each response; total N = 209 



fj 

1 

'i 

jj 



1 

.1 



1 

i 

j 

j 

i 

\ 



4 

I 



j 



<5 

.i 



i 









APPENDIX B 



.<» - f .I'l'.w'j 1. ji ... . 



TABLE III INDIVIDUAL PROGRAM USAGE 
BREAKDOWN BY ENROLLMENTt 

PURPOSE OF USING PROGRAMS 



On a limited 
Enrollment Trial Basis 



As an Integral With Controls 
Part of 
Curriculum 



4 Other 

Research Design Comment: Omit 



0-499 
500-999 
1,000-4,999 
5,000 + 

Totals 



19 

21 

52 

55 

152 



19 
11 
23 

20 

76 



4 

2 

18 

33 

58 



5 

6 



13 



6 

1 

11 



Enrollment I 2 



NUMBER OF PROGRAM USAGES IN EACH GRADE 

Grade 



0-499 
500-999 
1,000-4,999 
5,000 + 

Totals 



1 

2 

6 

8 



1 

2 

6 



3 

1 



4 


5 


6 


7 


8 


9 


10 


11 


IZMultiple 


1 


1 


2 


2 


1 


8 


17 


11 


15 


19 44 


i 

i 


1 


2 


1 


2 


8 


17 


16 


6 


12 33 


i 


5 


3 


5 


10 


22 


34 


24 


26 


22 91 


j 


15 


15 


12 


14 


19 


45 


36 


20 


14 96 


! 


22 


22 


20 


28 


59 117 


91 


71 


69 





INTENT 

Enrollment Remedial Instruction Regular Instruction Enrichment 

0-499 n 

500-999 6 

1,000-4,999 22 

5,000+ _26_ 

Totals 66 



33 

26 

69 

76 

201 



11 

10 

28 

19 

69 



TEACHER 

Enrollment As Active Supplement Available ForQuestions As ProctorOnly 

0-499 
500-999 
1,000-4999 
5,000 + 

Totals 



24 

20 

62 

57 

164 



23 

20 

37 

34 

116 



13 

15 

31 



0 

ERIC 



t Entries are n of users checking each response; total N = 282 

73 



nw» fff J« PJI llj , 1 1 1 



o 

ERIC 



TABLE HI (continued) 



CHOICE BY 



OnlyOne Publisher's Curriculum Subject Matter Subject Matter 
Enrol Iment Ava ilable Statement Coordinator Consultant Teacher 



TEST 



Enrollment Standardized 



Publisher 

Supplied 



Teacher 

Constructed 



0-499 
500-999 
1,000-4,999 
5,000 + 

Totols 



15 

15 

33 

41 



16 

12 

36 

26 



16 

15 

35 

45 



Other 

2 



Omit 



104 



93 



114 



5 

12 

19 



4' 

2 

11 

9 



30 



OVERALL EFFICIENCY 



Enrol Iment Useless 








Extremely 






Adequate 




Worthwhile 


Omit 


0-499 


i 


11 


10 


18 


5 


500-999 


2 


8 


17 


2 


3 


1,000-4,999 


6 


36 


24 


19 


11 


5,000 + 


6 


29 


31 


11_ 


18 


Totols 


16 


85 


83 


51 


41 



74 



0-499 


2 


ii 


6 


6 


27 


500-999 


4 


12 


6 


5 


20 -V 


1,000-4,999 


13 


21 


16 


8 


27 


5,000 + 


13 


21 


36 


24 


42 


Totols 


32 


67 


66 


44 


112 






EVALUATION 








Dark 


Somewhat 


Reasonably 




Too 

Advanced ' • 


Enrollment 


Ages 


Obsolete 


Up-to-Date 


Modern 


0-499 


— 




9 


31 




500-999 


— 


— 


10 


19 





1,000-4,999 


— 


5 


39 


47 





5,000 + 




11 


42 


36_ 


__ ^ 


Totols 


— 


16 


103 


135 


•i y 



1 



MiHSi 



mum 



mmmsmmsm m 



umumum mmuuujuuu 








TABLE in (continued) 



HELPFUL INFORMATION ABOUT A NEW PROGRAM WOULD INCLUDE 



Classification 
A Statement of 0 f 
Enrollment Objectives Content 



Sample 

Frames 



Criterion 

Frames 



Other 



0-499 
500-999 
1,000-4,999 
5,000 + 
Totols 



15 

15 

35 

57 



18 

18 

54 

57 



124 



149 



13 

13 

36 

45 

109 



4 

3 

22 

30 

59 



11 

14 

27 



75 



O 

ERIC 
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APPENDIX B 



TABLE IV INDIVIDUAL PROGRAM USAGE 
BREAKDOWN BY SUBJECT AREAt 



NUMBER OF COPIES USED 



Subject Area 


1-5 


6-23 


24-49 


50-99 


100-299 


300 + 


Totals 


% 


Math 


34 


38 


47 


21 


23 


8 


172 


62% 


Science 


— 


1 


1 


3 


3 


1 


9 


3% 


Foreign Language 


4 


3 


— 


— 


1 


2 


11 


4% 


English 


6 


5 


21 


17 


7 


3 


60 


21% 


Spelling 


3 


3 


1 


— 


4 


— 


11 


4% 


Social Studies 


— 


1 


— 


2 


4 


— 


7 


2% 


Other 


2 


— 


3 


— 


— 


— 


7 


2% 


Omit 


1 


1 


2 


1 


- 


- 


5 


2% 


Totals 


50 


52 


75 


44 


42 


14 


282 = 


total 


% 


17.7 


18.4 


26.6 


15.6 


14.9 


5.0 







LOCAL PROGRAMER'S TITLE 



Subject Area 


Omit 


Subject 

Teacher 


Curriculum 

Coordinator 


Audio-Visual 

Department 


Other 


Math 


150 


13 


2 





5 


Science 


2 


5 


— 


1 


1 


Foreign Language 


9 


1 


— 


— 




English 


48 


11 


— 


— 


1 


Spelling 


8 


1 


1 


— 


_ 


Social Studies 


1 


6 


— 


— 


— * 


Other 


4 


1 


— 







Omit 


— 


2 


— 


1 




Totals 


224 


40 


3 


2 


9 



tFor full questions— see Appendix A Individual Program Usage form. 




76 
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TABLE IV (continued) 



PROJECT DIRECTOR 



Subject Area 

Math 

Science 

Foreign Language 
English 
Spelling 
Social Studies 
Other 

Totals 

% 



Subject Area 

Math 

Science 

Foreign Language 
English 
Spelling 
Social Studies 
Other 

Totals 

% 



Audio- 

Subject Curriculum Visual 
Omit Teacher Coordinator Dept. Other Multiple 



80 43 10 


3 


28 


3 


4 1 


L 2 


- 


2 


— 


3 : 


3 2 


1 


1 


1 


31 « 


? 7 


— 


10 


3 


5 


1 3 


— 


2 




4 


2 1 


— 


— 


*** 


4 


2 


— 


1 


— 


136 66 25 


4 


44 


7 


48 23 8.9 


1.4 


15.6 


2.5 




PURPOSE 








Trial 


Integral 


Research 






Basis 


Curriculum 


Design 


Other 


Omit 


99 


44 


31 


12 


7 


2 


6 


2 






7 


3 


1 






30 


17 


16 


— 


3 


5 


1 


4 


— * 


1 


4 


2 


1 


— 




2 


2 


2 


1 





152 


76 


58 


13 


ii 


54 


27 


21 







i 



i 




77 



1 
















TABLE IV (continued) 



FORM OF PRESENTATION 



O 

ERIC 





Hori- 


Verti- 




Without 










Subject 


zontal 


cal Teaching Teaching Audi ■ 






Area Cards 


Text 


Text Machine Machine 


tory 


Other Omit 


p/e 


Math 


28 


87 


20 




16 






i 


9 


11 


Science - 


3 


1 


— 




1 









2 


2 


oreign Lang. — 


1 


1 


2 




— 




5 


_ 


— 


2 


English - 


43 


12 


2 




2 




_ 





— 


1 


Spelling — 


— 


— 


5 




1 







i 


— 


4 


iocial Stud. — 


3 


2 


2 














__ 




o"- 


- 


2 


2 




— 




- 


i 


1 


1* 


Totals — 


80 


107 


34 




20 




5 


3 


12 


21 


% 


28.4 


37.9 


12.1 




7.1 




1.8 


1.0 4.3 


7.5 






EACH GRADE 












Subject Area 


1 


2 3 4 


5 


6 


7 


8 


9 


10 11 


12 


Math 


5 


5 ! 


> 9 


11 


12 


12 


31 


78 


59 48 


43 


Science 


1 


2 2 


! 2 


2 


2 


2 


3 


3 


1 3 


4 


Foreign Language 4 


3 1 


L 1 


1 


2 


1 


2 


6 


7 3 


3 


English 


— 


r 


! 4 


3 


1 


10 


19 


25 


18 13 


16 


Spell ing 


3 


2 < 


1 4 


3 


1 


3 


2 


4 


1 1 


1 


Social Studies 


1 


1 1 


l 1 


1 


1 


_ 


2 


1 


1 2 





Other 


2 


2 2 


1 1 


- 


1 


— 


— 


— 


1 - 


1 


Total s 


17 


17 22 22 


22 


2Ql 28 


59 117 


90 71 


69 



MENTAL ABILITY 



Subject Area 


Above Average 


Average 


Below Average 


Math 


113 


98 


56 


Science 


3 


5 


1 


Foreign Language 


9 


5 


3 


English 


23 


39 


30 


Spelling 


5 


5 


3 


Social Studies 


4 


5 


3 


Other 


4 


6 


4 


Totals 


162 


166 


100 



78 



; 1 





















TABLE IV (continued) 



O 

ERIC 



SCHOOL SCHEDULE DEVOTED TO PROGRAMED INSTRUCTION 

Hours 



Subject Area 


a 


7 


7JS 


2 


3 


over 3 


Math 


24 


112 


9 


6 


3 


1 


Science 


4 


5 


— 


— 


— 


— 


Foreign Language 


5 


4 


— 


1 


— 


— 


English 


20 


29 


2 


1 


1 


— 


Spel ling 


7 


3 


— 


— 


— 


— 


Social Studies 


1 


4 


1 


— 


— 


1 


Other 


4 


2 


— 


— 


— 


— 


Totals 


70 


159 

Days 


12 


8 


4 


2 


Subject Area 


0 


I 


2 


3 


4 


5 


Math 


3 


6 


7 


4 


33 


98 


Science 


— 


— 


1 


— 


3 


5 


Foreign Language 


1 


2 


— 


1 


3 


4 


English 


2 


3 


10 


16 


9 


11 


Spelling 


— 


— 


— 


2 


2 


6 


Social Studies 


— 


2 


— 


— 


1 


4 


Other 


— 


1 


1 


4 


2 


1 




— 


— 


— 


— 




— .. ... 


Totals 


6 


14 

Weeks 


19 


28 


53 


129 


Subject Area 


I or /ess 2-3 


4*6 


7-75 


76-32 


over 32 


Math 


3 


3 


9 


12 


40 


79 


Science 


— 


2 


1 


— 


3 


3 


Foreign Language 


— 


— 


1 


1 


3 


4 


English 


3 


2 


11 


12 


18 


6 


Spelling 


— 


— 


3 


2 


2 


3 


Social Studies 


2 


1 


2 


- 


1 


1 


Other 


— 


2 


1 


1 


1 


1 




— — 


— 


— 


— 


■ 


- 


Totals 


8 


10 


28 


29 


69 


98 



79 



i 
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TABLE IV (continued) 



CHOICE 



Subject 


Only One 


Publ isher’s 


Curriculum 


Subject 

Matter 


Subject 

Matter 




Area 


Available 


Statement 


Coordinator Consultant Teacher Other 


Moth 


16 


49 


36 


26 


76 


24 


Science 


1 


— 


2 


4 


5 


2 


Foreign Long. _ 


3 


4 


3 


4 


1 


English 


12 


14 


14 


8 


/ 23 


5 


Spell ing 


2 


1 


5 


1 


2 


— 


Social Stud. 


— 


— 


— 


1 


6 


— 


Other 


1 


— 


3 


1 


1 


1 


Totals 


32 


"67" 


66 


44 


119 


33 



CONTENT EVALUATION 



Subject Area 



Dark Somewhat Reasonably Too 

Ages Obsolete Up-to-Date Modern Advanced 



Moth — 

Science — 

Foreign Language — 

English — 

Spelling — 

Social Studies — 

Other — 



13 



65 



3 

27 

4 
1 
1 



74 

9 

7 

30 

3 

4 

5 



Totals 



16 



103 



135 



TEST 



Subject Area 



Standardized 



Publisher 

Supplied 



Teacher 

Constructed 



Other 



Math 

Science 

Foreign Language 
English 
Spelling 
Social Studies 
Other 



69 

4 

2 

20 

5 

2 

1 



58 

2 

3 

24 

1 



Totals 



104 



3 

93 



63 

7 

5 

21 

7 

7 

3 



114 



2 

19 



80 




liiiiiii 



■ihttiiiMfitt 
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TABLE IV (concluded) 



Subject Area 


OVERALL EFFICIENCY 
Useless Adequate 


Extremely 

Worthwhile 


Math 


— 


8 51 


52 


28 


Science 


— 


— — 


6 


2 


Foreign Language 


— 


1 4 


2 


3 


English 


— 


3 21 


16 


12 


Spelling 


— 


3 4 


2 


1 


Soctal Studies 


— 


1 - 


4 


1 


Other 


__ 


- _3_ 




3 


Totals 




16 85 


83 


51 


Subject Area 


ENROLLMENT 
0-499 500-999 


1,000-4,999 


5,000 + 


Math 


36 


21 


53 


55 


Science 


— 


1 


4 


4 


Foreign Language 


— 


3 


3 


5 


English 


7 


6 


23 


23 


Spelling 


— 


1 


5 


5 


Social Studies 


— 


— 


3 


4 


Other 


2 


— 


4 


1 


Totals 


45 


33 


99 


97 






!■: ' 
it;' 







. V. • 
iv ; " 
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APPENDIX C 



TABLES OF SIGNIFICANCE 



Initial Source 


ft NorrUsers 


ft Users 


Chi Square 


Probability 


Newspapers & 


(of 1671) 


(of 209) 






Magazines 


526 


34 


20.5 


<.001 


Professional 










Publications 


1274 


150 


2.0 


Not Significant 


Colleagues 


206 


48 


> 17.9 


<.001 


Laymen 


17 


2 


Q.l 


Not Significant 


Salesmen 


298 


18 


>11.3 


<.001 




PUBLICATION 






Read Two Issues 


ft Non-Users 


ft Users 


Chi Square 


Probability 


A.V.I. 


578 


113 


> 30,3 


<.001 


AID 


102 


42 


> 51.9 


<.001 


ATB 


106 


53 


> 86.7 


<.001 


P.l. 


404 


125 


> 111.6 


<.001 


J. Ed. Psy. 


245 


69 


> 44.9 


<.001 


Galanter 


51 


30 


54.8 


<.001 


L. & G. 


255 


113 


> 336.6 


<.001 


Cram 


125 


40 


> 31.5 


<.001 


Stolurow 


57 


15 


>6.2 


<.05 


Markle 


51 


40 


> 104.4 


<.001 



PLANNED PURPOSE 





ft Non-Users 


ft Users 


Chi Square 


Probability 


Remedial 


482 


114 


> 54.2 


<.001 


Regular Instruction 


332 


142 


>217.9 


<.001 


Enrichment 


568 


125 


> 50.9 


<.001 



PLANNED FUNDING 





# NorrUsers 


ft Users 


Chi Square 


Probability 


Regular Budget 


1129 


187 


> 40.6 


<.001 


Extra Local Funds 


44 


6 


0.0 


Not Significant 


State 


39 


6 


0.1 


Not Significant 


Federal 


94 


18 


3.0 


Not Significant 



82 











APPENDIX C (continued) 



System Conference 
School Meeting 
Planning Group 
Program Director 
Staff to Workshop 
Consultant 
Obtain Samples 
Inform Lay Groups 



System Conference 
School Meeting 
Planning Group 
Program Director 
Staff to Workshop 
Consultant 
Obtain Samples 
Inform Lay Groups 



STEPS PLANNED 



# Non-Users 


» Users 


Chi Square 


Probability 


136 


16 


< .5 


Not Significant 


137 


13 


<4 


Not Significant 


315 


36 


.32 


Not Significant 


34 


6 


.3 


Not Significant 


198 


46 


> 17.0 


<.001 


306 


57 


>9.6 


<.01 


591 


90 


>5.2 


<.05 


230 


66 


> 44.4 


<.001 


STEPS TAKEN 






# Non-Users 


# Users 


Chi Square 


Probability 


157 


46 


> 30. 7 , 


<.001 


218 


52 


> 21.1 


<.001 


98 


67 


> 152.4 


<.001 


21 


18 


>45.9 


<.001 


224 


68 


>54.1 


<.001 


180 


71 


>82.7 


<.001 


580 


171 


> 16.45 


<.001 


137 


99 


> 248.5 


<.001 
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